Exploring the feasibility of conversational diagnostic AI in a real-world clinical study

2026-03-11 · Source: The latest research from Google · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Emerging Technologies & Innovation, Data Science & Analytics · Depth: Advanced, long

Summary

Google Research and Google DeepMind, in partnership with Beth Israel Deaconess Medical Center (BIDMC), conducted a prospective, single-center feasibility study of AMIE, their conversational medical AI. This study, detailed in a paper published March 11, 2026, assessed AMIE's ability to gather pre-visit clinical history from 100 adult patients before ambulatory primary care appointments. The AI-driven text chats were supervised by a physician via live video, with zero safety stops required. Results showed that patients' trust in AI increased after interacting with AMIE, and clinical evaluators rated AMIE's differential diagnosis (DDx) and overall management plan (Mx plan) quality on par with Primary Care Providers (PCPs). AMIE achieved 90% top-7 diagnostic accuracy and 56% top-1 accuracy, though PCPs outperformed AMIE in practicality and cost-effectiveness of Mx plans due to access to EHR and physical exam capabilities.

Key takeaway

For AI Scientists developing clinical diagnostic tools, this study demonstrates that supervised conversational AI can safely and effectively gather patient history in real-world settings. You should focus on integrating multimodal inputs and EHR access to enhance AI's practicality and cost-effectiveness in management plans, addressing current limitations. Consider designing future studies with controlled comparisons to quantify the impact of AI interventions against baseline workflows.

Key insights

Supervised conversational AI for pre-visit history taking is feasible, safe, and well-received by patients and clinicians.

Principles

• Real-world clinical assessment is crucial for AI translation.
• Physician oversight enhances AI safety in clinical settings.
• AI can improve patient-provider interaction dynamics.

Method

Patients interacted with AMIE via secure web-link for pre-visit history taking, overseen by a physician. AMIE generated transcripts and summaries for PCPs, and clinical evaluators blinded-rated AMIE and PCP performance.

In practice

• Implement AI for pre-visit patient data collection.
• Integrate live physician supervision for AI safety.
• Use AI to shift clinical visits to data verification.

Topics

• Conversational AI
• Medical AI
• Clinical Diagnostics
• AI Safety
• Feasibility Study

Best for: AI Scientist, Research Scientist, AI Researcher, Machine Learning Engineer, AI Ethicist

Related on AIssential

• ChatHealthAI: Aligning Electronic Health Record Representations with Large Language Models for Grounded Clinical Reasoning — ChatHealthAI integrates EHR representations with LLMs for grounded, interpretable clinical reasoning and accurate patient prediction.
• Collaborating on a nationwide randomized study of AI in real-world virtual care — A nationwide randomized study will rigorously evaluate conversational AI in real-world virtual healthcare settings.
• Possible or Definite? A Benchmark for Evaluating Diagnostic Uncertainty Preservation in Clinical Text — Large language models poorly preserve diagnostic uncertainty in clinical text, posing a significant safety risk.
• How good are ‘AI doctors’ — and will they take over medicine? — Advanced AI models are matching or exceeding human diagnostic accuracy in specific medical contexts, but real-world application is complex.
• Real-world safety and harms from patient-facing LLMs — Real-world data on AI chatbot harms in health is limited but growing, highlighting mental health risks.
• AIPatient Arena: EHR-grounded evaluation of large language models in end-to-end clinical consultation workflows — AIPatient Arena reveals LLMs need process-based evaluation beyond final answers for clinical readiness, highlighting interaction failures.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by The latest research from Google.