LAMAR at MedExACT 2026: Agreement-Driven Large Language Model Ensembles for Clinical Decision Extraction from Discharge Summaries

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Health & Medical Research · Depth: Expert, quick

Summary

The LAMAR system, an ensemble of Qwen3.5-4B language models, was developed to extract medical decisions from discharge summaries within the MedDec dataset. These models were specifically trained to annotate summaries using inline XML-like tags. The development incorporated three distinct training strategies: dynamic fine-tuning, reinforcement learning, and pseudo-label augmentation. By leveraging inter-model agreement to combine predictions, the LAMAR system significantly enhanced its performance across various evaluation metrics. It achieved an overall F1 score of 0.5942 and secured the second position on the test leaderboard. Furthermore, the system demonstrated stable performance across different demographic groups, indicating a degree of fairness for underrepresented populations.

Key takeaway

For NLP Engineers developing clinical decision support systems, this research suggests that combining multiple Qwen3.5-4B models through inter-model agreement can significantly boost extraction accuracy. You should consider implementing diverse training strategies like dynamic fine-tuning and pseudo-labeling to improve model robustness. Furthermore, rigorously evaluate your system's performance across various demographic groups to ensure fairness and mitigate bias in critical healthcare applications.

Key insights

An agreement-driven ensemble of Qwen3.5-4B LLMs effectively extracts clinical decisions, achieving high F1 and demographic fairness.

Principles

• Ensemble agreement improves LLM extraction.
• Diverse training strategies enhance performance.
• Evaluate LLM fairness across demographics.

Method

Qwen3.5-4B models were trained with dynamic fine-tuning, reinforcement learning, and pseudo-label augmentation to annotate discharge summaries with XML-like tags. Predictions were combined via inter-model agreement.

In practice

• Use Qwen3.5-4B for clinical text.
• Apply ensemble methods for robustness.
• Test LLM output for demographic bias.

Topics

• Large Language Models
• Clinical Decision Support
• Ensemble Learning
• Qwen3.5-4B
• Discharge Summaries
• AI Fairness

Best for: Research Scientist, AI Scientist, NLP Engineer, Machine Learning Engineer

Related on AIssential

• ClinicalMC: A Benchmark for Multi-Course Clinical Decision-Making with Large Language Models — Evaluating LLMs in evolving multi-course clinical scenarios is crucial for real-world healthcare deployment.
• AI-assisted Protocol Information Extraction For Improved Accuracy and Efficiency in Clinical Trial Workflows — AI-assisted RAG significantly boosts clinical trial protocol data extraction accuracy and efficiency over standalone LLMs.
• Domain-Adapted Small Language Models for Reliable Clinical Triage — Domain-adapted SLMs can provide reliable, privacy-preserving ESI decision support, outperforming larger proprietary models.
• L2D-Clinical: Learning to Defer for Adaptive Model Selection in Clinical Text Classification — Adaptive model selection in clinical text classification improves accuracy by deferring to LLMs based on uncertainty.
• A Large-Scale Multi-Dimensional Empirical Study of LLMs for Conversation Summarization — OmniCSEval offers a robust, multi-dimensional benchmark for evaluating LLMs in conversation summarization across diverse real-world scenarios.
• From Hazard Functions to Language Space: Cox-Supervised Distillation of Survival Risk into a Large Language Model — Large language models can internalize and predict survival risk from structured clinical data via text-based distillation.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.