Domain-Adapted Small Language Models for Reliable Clinical Triage

2026-04-29 · Source: Computation and Language · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Medical Devices & Health Technology · Depth: Expert, quick

Summary

A study evaluated open-source small language models (SLMs) as privacy-preserving decision-support tools for clinical triage, specifically for Emergency Severity Index (ESI) assignment. Researchers compared multiple SLMs using diverse prompting pipelines, finding that clinical vignettes—concise summaries of triage narratives—produced the most accurate predictions. The Qwen2.5-7B SLM emerged as the optimal choice due to its balance of accuracy, stability, and computational efficiency. Through extensive domain adaptation using expert-curated and silver-standard pediatric triage data, fine-tuned Qwen2.5-7B models significantly reduced discordance and clinically significant errors. These fine-tuned SLMs outperformed all baseline SLMs and even advanced proprietary large language models like GPT-4o, demonstrating the viability of institution-specific SLMs for reliable ESI decision support.

Key takeaway

For emergency department administrators and NLP engineers developing clinical decision support, this research indicates that institution-specific, fine-tuned small language models like Qwen2.5-7B can offer more accurate and privacy-preserving ESI triage assistance than larger, proprietary models. You should consider investing in domain adaptation and targeted fine-tuning with clinical vignettes to improve triage accuracy and reduce mistriage.

Key insights

Domain-adapted SLMs can provide reliable, privacy-preserving ESI decision support, outperforming larger proprietary models.

Principles

• Targeted fine-tuning improves SLM performance.
• Clinical vignettes enhance triage prediction accuracy.

Method

The study involved systematically comparing SLMs across prompting pipelines, followed by large-scale domain adaptation using expert-curated and silver-standard pediatric triage data for fine-tuning.

In practice

• Use Qwen2.5-7B for clinical triage applications.
• Prioritize fine-tuning over complex inference strategies.

Topics

• Emergency Severity Index
• Small Language Models
• Clinical Triage
• Qwen2.5-7B
• Domain Adaptation

Best for: NLP Engineer, AI Scientist, Machine Learning Engineer, Research Scientist

Related on AIssential

• When Large Language Models Fail in Healthcare: Evaluating Sensitivity to Prompt Variations — Healthcare LLMs are not intrinsically safe, exhibiting high sensitivity to prompt variations that can lead to harmful clinical outputs.
• Fine-Tuning Qwen3.5 — Fine-tuning small vision-language models like Qwen3.5-0.8B on domain-specific datasets significantly improves specialized task performance.
• Small LLMs for Biomedical Claim Verification: Cost-Effective Fine-Tuning, Structural Dataset Shortcuts, and Cross-Domain Generalization — Small, QLoRA-fine-tuned LLMs can surpass larger proprietary models for specialized biomedical claim verification tasks.
• Trade-offs in Medical LLM Adaptation: An Empirical Study in French QA — SFT offers the best performance-efficiency trade-off for medical MCQA, while CPT benefits OEQA but with caveats.
• From Hazard Functions to Language Space: Cox-Supervised Distillation of Survival Risk into a Large Language Model — Large language models can internalize and predict survival risk from structured clinical data via text-based distillation.
• L2D-Clinical: Learning to Defer for Adaptive Model Selection in Clinical Text Classification — Adaptive AI-to-AI deferral improves clinical text classification by leveraging complementary model strengths and reducing LLM costs.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computation and Language.