How good are ‘AI doctors’ — and will they take over medicine?

2026-06-03 · Source: Machine learning : nature.com subject feeds · Field: Health & Wellbeing — Medical Devices & Health Technology, Clinical Care & Medical Practice, Health & Medical Research · Depth: Advanced, quick

Summary

Recent studies highlight the evolving capabilities of AI tools in medical diagnosis, though their readiness to fully replace physicians remains debated. An April 2026 Science study found OpenAI's o1 large language model (LLM) achieved 67% correct or near-correct diagnoses in emergency department cases, surpassing human physicians' 50-55% accuracy when reviewing recorded patient information. Separately, Google Research's Articulate Medical Intelligence Explorer (AMIE) demonstrated similar diagnostic performance to human clinicians, with its top three suggestions including the correct diagnosis in 75% of cases and being the top suggestion in 56% after text-based patient conversations. While AI already handles tasks like note-taking and prescription renewals, researchers emphasize that real-world medical complexity, involving direct patient interaction and nuanced situations, still poses significant challenges for these systems.

Key takeaway

For healthcare administrators and medical professionals evaluating AI integration, recognize that advanced LLMs like OpenAI's o1 and Google's AMIE demonstrate strong diagnostic capabilities in specific scenarios. Your focus should be on using AI as a powerful diagnostic aid for initial assessments and structured data analysis. It is not a complete replacement for human physicians. Human interaction and nuanced clinical judgment remain indispensable for handling complex patient cases and formulating practical, cost-effective treatment plans.

Key insights

Advanced AI models are matching or exceeding human diagnostic accuracy in specific medical contexts, but real-world application is complex.

Principles

• AI excels with structured patient data.
• Direct patient interaction remains a human strength.
• AI diagnostic capabilities are rapidly advancing.

Method

AI systems like OpenAI's o1 analyze recorded patient data, while Google's AMIE converses via text to collect histories and suggest diagnoses, often before human physician interaction.

In practice

• Automate note-taking and prescription renewals.
• Utilize AI for initial patient history collection.
• Integrate AI for diagnostic support in EDs.

Topics

• Medical AI
• Large Language Models
• Diagnostic AI
• Clinical Trials
• Emergency Medicine
• Healthcare Technology

Best for: Executive, AI Product Manager, Investor, Research Scientist, AI Scientist, Domain Expert

Related on AIssential

• AI shows its skills in the emergency room — An older OpenAI model surpassed human doctors in diagnosing emergency room cases using only text data.
• Vivek Natarajan of Google DeepMind at RAAIS 2026 — Building useful AI in expert domains requires moving beyond benchmarks to real-world clinical interaction and new knowledge generation.
• Six-in-ten doctors have used AI in clinical practice – survey - Irish Medical Times — AI is rapidly integrating into clinical practice, yet doctors require significant training and guardrails for safe, ethical use.
• Can AI Chatbots Reason Like Doctors? — Large language models demonstrate potential in clinical reasoning, but require careful integration and standardized evaluation in medical workflows.
• Exploring the feasibility of conversational diagnostic AI in a real-world clinical study — Supervised conversational AI for pre-visit history taking is feasible, safe, and well-received by patients and clinicians.
• AI for Interoperability in Health Care: Philips’s Carla Goulart Peron — AI in healthcare augments clinicians to expand access and improve diagnostics, with interoperability identified as the most impactful global capability.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Machine learning : nature.com subject feeds.