LiteOdyssey: A Lightweight Reasoning AI Agent for Interpretable Rare-Disease Diagnosis

2026-07-11 · Source: cs.AI updates on arXiv.org · Field: Health & Wellbeing — Medical Devices & Health Technology, Health & Medical Research, Medical Specialties & Subspecialties · Depth: Expert, extended

Summary

LiteOdyssey is a lightweight AI agent designed for interpretable rare-disease diagnosis, utilizing a single reasoning-capable language model guided by a human-AI collaborative diagnostic policy and augmented with freely available biomedical tools. This framework achieved high diagnostic performance, demonstrating an overall disease Recall@1 of 59.3% across 1,243 cases from the LIRICAL (n=370) and PhenoPacket Store (n=873) benchmarks. Notably, on the more challenging PhenoPacket subset, LiteOdyssey attained 60.7% Recall@1, significantly outperforming the baseline GPT-5.4 model's 10.7% without tools. The system's effectiveness extends to a private cohort of 515 real-world Undiagnosed Diseases Network patients, where it showed a Recall@1 of 20.4% versus 16.7% for the baseline. This performance was achieved without extensive fine-tuning or large retrieval databases, emphasizing its deployability and transparency through auditable reasoning traces.

Key takeaway

For AI scientists developing diagnostic systems for rare diseases, LiteOdyssey demonstrates that focusing on structured reasoning and external policies, rather than just model scale or fine-tuning, can yield high accuracy. You should consider implementing human-feedback-driven policy iteration and integrating public biomedical tools to create interpretable, deployable, and auditable solutions, especially in resource-constrained clinical settings. This approach offers a path to high performance with reduced infrastructure burden.

Key insights

Structured reasoning with public biomedical tools enables highly accurate rare-disease diagnosis using a single lightweight AI agent.

Principles

• Diagnostic performance relies on reasoning organization.
• External policies enhance model capabilities.
• Lightweight systems achieve high accuracy.

Method

Policy Iteration with Human Feedback (PIHF) develops a natural-language diagnostic policy. A frozen model executes the policy, human experts review traces and model critiques, guiding policy revisions without weight changes.

In practice

• Integrate public biomedical tools into AI workflows.
• Implement PIHF for transparent policy development.
• Use reasoning traces for clinical auditability.

Topics

• Rare Disease Diagnosis
• AI Agents
• Language Models
• Policy Iteration with Human Feedback
• Biomedical Tools
• Clinical Interpretability

Best for: AI Scientist, Research Scientist, Domain Expert

Related on AIssential

• 😺 OpenAI found 18 rare diseases — AI models can significantly aid human clinicians in diagnosing complex rare diseases by generating evidence-linked leads.
• A safety-oriented hypothetico-deductive framework for AI-assisted differential diagnosis — Engineering diagnostic AI as a safety-oriented reasoning framework provides safer, more transparent clinical decision support.
• ArogyaSutra: A Multi-Agent Framework for Multimodal Medical Reasoning in Indic Languages — A multi-agent framework and multilingual multimodal dataset significantly improve medical reasoning in low-resource Indic language healthcare.
• Statistics, Not Scale: Modular Medical Dialogue with Bayesian Belief Engine — Strictly separating language processing from probabilistic reasoning in medical AI yields superior, auditable, and private diagnostic systems.
• VAIDYA: Validated Agents for Intelligent Diagnosis and Yielded Analysis — New benchmarks and a structured AI framework improve LLM medical diagnosis evaluation by simulating real-world clinical interaction and reasoning.
• SymptomAI: Towards a conversational AI agent for everyday symptom assessment — Conversational AI can achieve clinician-preferred and more accurate differential diagnoses in real-world symptom assessment.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.AI updates on arXiv.org.