A Proactive EMR Assistant for Doctor-Patient Dialogue: Streaming ASR, Belief Stabilization, and Preliminary Controlled Evaluation

2026-04-16 · Source: cs.AI updates on arXiv.org · Field: Health & Wellbeing — Medical Devices & Health Technology, Clinical Care & Medical Practice · Depth: Expert, long

Summary

A new end-to-end proactive Electronic Medical Record (EMR) assistant has been developed to provide real-time consultation support during doctor-patient dialogues, moving beyond passive post-consultation note generation. This system integrates streaming Automatic Speech Recognition (ASR), punctuation restoration, stateful information extraction, belief stabilization, objectified retrieval, action planning, and replayable report generation. Evaluated in a preliminary controlled setting using ten streamed doctor-patient dialogues and a 300-query retrieval benchmark, the full system achieved a state-event F1 of 0.84, retrieval Recall@5 of 0.87, and end-to-end pilot scores of 83.3% coverage, 81.4% structural completeness, and 80.0% risk recall. Ablation studies suggest that both punctuation restoration and belief stabilization significantly improve downstream extraction, retrieval, and action selection within this pilot, demonstrating the technical coherence and directional support of the online architecture under tightly controlled conditions.

Key takeaway

For NLP engineers developing real-time clinical documentation systems, you should prioritize integrating streaming ASR with robust punctuation restoration and belief stabilization. This architecture, demonstrated to improve extraction, retrieval, and action selection in pilot studies, offers a concrete baseline for building proactive EMR assistants. Your next steps should involve validating this pipeline on larger, more diverse prospective clinical data to assess real-world performance and clinical utility.

Key insights

Proactive EMR assistants require streaming ASR, belief stabilization, and objectified retrieval for real-time consultation support.

Principles

• Punctuation is critical for downstream NLP tasks.
• Stabilize raw LM outputs for reliable clinical decisions.
• Objectify content for enhanced retrieval quality.

Method

The system uses streaming ASR, punctuation restoration, stateful extraction, belief stabilization, hybrid retrieval, and POMDP-lite action planning to maintain a dynamic state and suggest next actions.

In practice

• Implement punctuation restoration after ASR.
• Apply temperature scaling and exponential smoothing to belief states.
• Anchor medical objects for traceability and replay.

Topics

• Proactive EMR Assistant
• Streaming ASR
• Punctuation Restoration
• Belief Stabilization
• Hybrid Retrieval

Best for: NLP Engineer, AI Scientist, Machine Learning Engineer, Research Scientist

Related on AIssential

• When Multiple Scripts Matter: Evaluating ASR in Clinical Settings — Multiscript variability in non-English clinical ASR requires specialized evaluation and script unification during training for accurate performance.
• User-Aware Active Knowledge Acquisition for Emotional Support Dialogue — UKA uses active learning and Theory-of-Mind uncertainty to improve emotional support dialogue by efficiently acquiring user-aligned knowledge.
• EMSDialog: Synthetic Multi-person Emergency Medical Service Dialogue Generation from Electronic Patient Care Reports via Multi-LLM Agents — Synthetic multi-speaker EMS dialogues generated from ePCRs enhance conversational diagnosis prediction accuracy and stability.
• Retrieval-Augmented Generation for Clinical Question Answering in Portuguese Drug Leaflets: Benefits and Limitations — RAG improves factual recall in clinical Q&A but needs adaptive control for complex reasoning to avoid anchoring bias.
• EHRNote-ChatQA: A Benchmark for Evidence-Grounded Multi-Turn Clinical Question Answering over Longitudinal Discharge Summaries — EHRNote-ChatQA benchmarks LLMs on evidence-grounded, multi-turn clinical QA over longitudinal discharge summaries, revealing current limitations.
• Evaluate Clinical ASR Models Faster with Agent Skills and NVIDIA Nemotron Speech — AI agent-guided synthetic data generation with pronunciation control enables rapid, repeatable clinical ASR evaluation and targeted model improvement.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.AI updates on arXiv.org.