EMSDialog: Synthetic Multi-person Emergency Medical Service Dialogue Generation from Electronic Patient Care Reports via Multi-LLM Agents

2026-04-10 · Source: cs.CL updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Natural Language Processing · Depth: Expert, quick

Summary

EMSDialog introduces a novel dataset of 4,414 synthetic multi-speaker Emergency Medical Service (EMS) conversations, generated from real-world Electronic Patient Care Reports (ePCRs). This dataset addresses the scarcity of multi-party medical dialogue corpora needed for conversational diagnosis prediction, which requires models to track evolving evidence in streaming clinical conversations. The generation pipeline utilizes multi-LLM agents, grounded in ePCRs and topic flow, to iteratively plan, generate, and self-refine dialogues with rule-based factual and topic flow checks. EMSDialog is annotated with 43 diagnoses, speaker roles, and turn-level topics. Human and LLM evaluations confirm the high quality and realism of the dataset using both utterance- and conversation-level metrics. Training with EMSDialog significantly improves the accuracy, timeliness, and stability of EMS conversational diagnosis prediction models.

Key takeaway

For research scientists developing conversational AI for medical diagnosis, EMSDialog offers a critical resource to overcome limitations of existing dyadic datasets. You should consider integrating this synthetic multi-speaker dataset into your training regimens to improve model accuracy, timeliness, and stability in tracking evolving evidence within complex clinical conversations. This approach can lead to more robust and reliable diagnostic prediction systems.

Key insights

Synthetic multi-speaker EMS dialogues generated from ePCRs enhance conversational diagnosis prediction accuracy and stability.

Principles

• Multi-agent LLM pipelines can generate high-quality synthetic data.
• ePCRs provide a robust grounding for medical dialogue synthesis.
• Topic flow and factual checks are crucial for dialogue realism.

Method

An ePCR-grounded, topic-flow-based multi-agent generation pipeline iteratively plans, generates, and self-refines dialogues using rule-based factual and topic flow checks to create synthetic multi-speaker EMS conversations.

In practice

• Use multi-LLM agents for complex data generation.
• Ground synthetic data generation in real-world reports.
• Incorporate rule-based checks for factual accuracy.

Topics

• EMSDialog Dataset
• Multi-LLM Agents
• Conversational Diagnosis Prediction
• Electronic Patient Care Reports
• Multi-speaker Dialogue Generation

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• Beyond one-on-one: Authoring, simulating, and testing dynamic human-AI group conversations — DialogLab bridges scripted and generative AI dialogue for complex multi-party human-AI conversations.
• A Proactive EMR Assistant for Doctor-Patient Dialogue: Streaming ASR, Belief Stabilization, and Preliminary Controlled Evaluation — Proactive EMR assistants require streaming ASR, belief stabilization, and objectified retrieval for real-time consultation support.
• ArogyaSutra: A Multi-Agent Framework for Multimodal Medical Reasoning in Indic Languages — Multimodal medical reasoning in Indic languages is enhanced by an actor-critic framework with tool grounding and dual-memory.
• ATPO: Adaptive Tree Policy Optimization for Multi-Turn Medical Dialogue — ATPO uses uncertainty-aware adaptive tree search and efficient execution to optimize multi-turn medical dialogue LLMs.
• Efficient ASR Training with Conversations that Never Happened — LLM-generated synthetic conversations, synthesized via TTS, effectively augment scarce real data for ASR training.
• Dial HEALTHDIAL for Advice: A Multilingual and Multi-Parallel Spoken Dialogue Dataset for Knowledge-Grounded Information Seeking — Creating large-scale, multilingual, multi-parallel spoken dialogue datasets is methodologically challenging, revealing performance disparities.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CL updates on arXiv.org.