HealthCraft: A Reinforcement Learning Safety Environment for Emergency Medicine

2026-05-23 · Source: cs.LG updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems, Medical Devices & Health Technology · Depth: Expert, extended

Summary

HealthCraft is the first public reinforcement learning environment designed to safely evaluate large language models (LLMs) in emergency medicine clinical workflows. Adapting the Corecraft architecture, it features a FHIR R4 world state with 14 entity types and 3,987 seed entities, alongside 24 MCP tools for interaction. A dual-layer rubric incorporates a hard safety gate, zeroing reward if any of the 515 safety-critical criteria are violated across its 2,255 binary criteria. The benchmark comprises 195 tasks across six categories. V8 results indicate Claude Opus 4.6 achieved a Pass@1 of 24.8% and GPT-5.4 achieved 12.6%, with safety-failure rates of 27.5% and 34.0% respectively. Notably, performance on multi-step workflows collapsed to 1.0% for Claude and 0.0% for GPT-5.4. The project also transparently documented six infrastructure bugs, whose resolution between pilots v2 and v8 significantly impacted model rankings, underscoring the importance of evaluation infrastructure fidelity.

Key takeaway

For Machine Learning Engineers developing LLMs for clinical decision support, HealthCraft's findings indicate that current frontier models like Claude Opus 4.6 and GPT-5.4 exhibit unacceptably high safety-failure rates (27.5% and 34.0%) and collapse on multi-step workflows. You must prioritize robust safety mechanisms, such as hard safety gates, and rigorous, high-fidelity RL environments during development and evaluation. Do not rely solely on static QA benchmarks; they fail to capture critical trajectory-level safety issues.

Key insights

HealthCraft provides a safety-gated RL environment for evaluating LLMs in emergency medicine, revealing current models' significant safety failures.

Principles

• Clinical safety requires hard gates, not tradeoffs.
• Infrastructure fidelity impacts model evaluation results.
• Static QA benchmarks miss critical safety failures.

Method

HealthCraft uses a Docker-bundled RL environment with a FHIR R4 Postgres world state, a FastMCP tool server, and a task engine. It applies a dual-layer rubric with a hard safety gate for reward calculation.

In practice

• Use FHIR R4 for clinical environment grounding.
• Implement hard safety gates in critical systems.
• Prefer "world_state" verification over "llm_judge".

Topics

• Reinforcement Learning
• Large Language Models
• Clinical Decision Support
• FHIR R4
• AI Safety
• Emergency Medicine Benchmarks

Code references

• GOATnote-Inc/healthcraft

Best for: Research Scientist, CTO, VP of Engineering/Data, AI Scientist, Machine Learning Engineer, AI Security Engineer

Related on AIssential

• PSEBench: A Controllable and Verifiable Benchmark for Evaluating LLMs in Patient Safety Event Triage — LLM evaluation for high-stakes patient safety triage requires policy-grounded, interactive benchmarks beyond static classification.
• ‘Unbelievably dangerous’: experts sound alarm after ChatGPT Health fails to recognise medical emergencies — ChatGPT Health frequently misdiagnoses medical emergencies and suicidal ideation, posing significant patient safety risks.
• When eight popular large language models (LLMs) answer common parent questions about early maxillary expansion, how reliable are they—and how readable is what they say? — LLM performance in healthcare varies significantly across accuracy, comprehensiveness, and readability, with no single model excelling in all.
• MedPanel AI - Multi-Agent Clinical AI — Multi-agent systems, particularly with adversarial agents, enhance diagnostic safety by simulating human clinical reasoning and challenging initial conclusions.
• Active Evidence-Seeking and Diagnostic Reasoning in Large Language Models for Clinical Decision Support — LLMs exhibit a substantial performance gap in active, multi-turn clinical diagnosis compared to static, full-context evaluations.
• Real-world safety and harms from patient-facing LLMs — Real-world data on AI chatbot harms in health is limited but growing, highlighting mental health risks.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.LG updates on arXiv.org.