H-AdminSim: A Multi-Agent Simulator for Realistic Hospital Administrative Workflows with FHIR Integration

2025-03-22 · Source: cs.AI updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems, Healthcare Systems & Policy · Depth: Expert, extended

Summary

H-AdminSim is a novel, comprehensive multi-agent simulation framework designed to evaluate Large Language Model (LLM)-based automation in hospital administrative workflows, specifically focusing on first-visit outpatient scenarios. It integrates realistic synthetic data generation, multi-agent dialogue simulation for patient intake and appointment scheduling, and Fast Healthcare Interoperability Resources (FHIR) for unified data exchange across heterogeneous hospital settings. The framework simulates hospitals of varying scales (primary, secondary, tertiary) with diverse patient backgrounds, including 194 disease-symptom pairs across nine internal medicine specialties. Quantitative evaluation uses detailed rubrics for tasks like department assignment and appointment scheduling, including rescheduling and cancellation. Experiments with GPT-5 Mini, GPT-5 Nano, and Gemini 2.5 Flash demonstrate that while tool-based scheduling is stable, patient intake, particularly department assignment accuracy, remains a significant bottleneck for LLM performance.

Key takeaway

For AI Scientists and Machine Learning Engineers developing LLM-based healthcare solutions, you should prioritize robust tool-based scheduling pipelines for appointment management, as pure LLM reasoning is significantly less reliable. Focus your research and development efforts on enhancing LLM performance in patient intake and department assignment, as this remains the primary challenge. Consider integrating FHIR standards early in your simulation and deployment strategies to ensure interoperability and scalability across diverse hospital environments.

Key insights

H-AdminSim provides a FHIR-integrated multi-agent simulation for evaluating LLM automation in complex hospital administrative workflows.

Principles

• Realistic simulation requires diverse patient profiles and hospital scales.
• Tool-based scheduling outperforms pure LLM reasoning.
• Patient intake is a primary bottleneck for LLM-driven administration.

Method

H-AdminSim uses hierarchical synthetic data generation, multi-agent dialogue for patient intake and scheduling, and FHIR integration for real-time hospital information system updates, evaluated via detailed rubrics.

In practice

• Use tool-calling for LLM-based scheduling to ensure stability.
• Prioritize improving LLM performance in patient intake dialogues.
• Leverage FHIR for interoperable healthcare data simulation.

Topics

• Multi-Agent Simulation
• Hospital Administrative Workflows
• FHIR Integration
• LLM Evaluation
• Patient Intake

Best for: AI Scientist, Machine Learning Engineer, Research Scientist

Related on AIssential

• AeroSpectra Sentinel: An Auditable LLM Prompt-Chaining Decision-Support Workflow for Acute Asthma Risk Assessment from Respiratory Sounds and Clinical Signals — An auditable LLM prompt-chaining workflow can enhance acute asthma risk assessment by integrating respiratory sounds and clinical data.
• Are LLMs Ready to Assist Physicians? PhysAssistBench for Interactive Doctor-Patient-EHR Assistance — Medical LLM assistance requires coordinated capabilities across knowledge, communication, and systems, not isolated gains.
• Build an agentic AI healthcare claims pipeline with Amazon Bedrock and AWS HealthLake — Agentic AI combined with intelligent document processing can automate complex healthcare claims validation and FHIR resource creation.
• MedPanel AI - Multi-Agent Clinical AI — Multi-agent systems, particularly with adversarial agents, enhance diagnostic safety by simulating human clinical reasoning and challenging initial conclusions.
• PSEBench: A Controllable and Verifiable Benchmark for Evaluating LLMs in Patient Safety Event Triage — LLM evaluation for high-stakes patient safety triage requires policy-grounded, interactive benchmarks beyond static classification.
• An Infectious Disease Spread Simulation Based on Large Language Model Decision Making — LLMs can simulate nuanced human health behaviors in spatially-grounded agent-based models, revealing demographic disparities.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.AI updates on arXiv.org.