Have LLMs improved patient outcomes?

2025-06-07 · Source: Marcus on AI · Field: Health & Wellbeing — Health & Medical Research, Medical Devices & Health Technology · Depth: Intermediate, quick

Summary

Cardiologist and author Eric Topol's latest review concludes that there is "very little evidence for LLMs benefiting patients or doctors for health outcomes," beyond administrative tasks. This finding aligns with a recent editorial in Nature Medicine, which also indicates a lack of direct evidence for large language models improving health outcomes. The analysis supports earlier warnings about the unsupervised use of LLMs for medical advice, suggesting that while AI holds future promise for medicine, current domain-general chatbots may not be suitable for direct clinical applications or patient-facing medical guidance. The consensus points to a gap between the potential of AI in healthcare and the current demonstrable impact of LLMs on patient and physician health outcomes.

Key takeaway

For AI Product Managers developing healthcare solutions, you should prioritize applications that address administrative burdens rather than direct patient health outcomes. The current evidence suggests that general-purpose LLMs lack the validated efficacy for clinical use, indicating a need for more specialized, evidence-based AI tools before widespread adoption in patient care. Focus on demonstrating clear, measurable benefits in specific, non-clinical workflows.

Key insights

Current LLMs show minimal evidence of directly improving patient or doctor health outcomes.

Principles

• AI's medical benefits are largely administrative.
• Domain-general chatbots are not yet clinically ready.

In practice

• Review LLM applications for administrative efficiency.
• Exercise caution with LLMs for direct patient care.

Topics

• LLMs in Medicine
• Patient Outcomes
• Medical AI Implementation
• Chatbot Medical Advice
• Eric Topol

Best for: AI Product Manager, Research Scientist, AI Scientist, AI Ethicist

Related on AIssential

• Health-care AI is here. We don’t know if it actually helps patients. — Despite rapid AI adoption in healthcare, its impact on patient outcomes is largely unmeasured.
• Are LLMs Ready to Assist Physicians? PhysAssistBench for Interactive Doctor-Patient-EHR Assistance — Medical LLM assistance requires coordinated capabilities across knowledge, communication, and systems, not isolated gains.
• Benchmarking GPT Models for Conversational AI Systems: Can AI Read a Doctor’s Notes? — LLMs struggle with medical abbreviation expansion, especially context-dependent ones, despite larger models performing better.
• Real-world safety and harms from patient-facing LLMs — Real-world data on AI chatbot harms in health is limited but growing, highlighting mental health risks.
• Towards AI-augmented decision making in psychiatry — AI-augmented decision-making can standardize care for heterogeneous psychiatric disorders.
• v333: Proceedings of CHIL 2026 — Machine learning, particularly LLMs, is rapidly expanding its utility across diverse clinical and administrative healthcare applications.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Marcus on AI.