Team Aurum at MedExACT 2026@ACL: Data Augmentation and Clinical Longformer Fine-Tuning for Medical Decision Extraction

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Health & Wellbeing — Artificial Intelligence & Machine Learning, Clinical Care & Medical Practice, Medical Devices & Health Technology · Depth: Advanced, quick

Summary

Team Aurum's system for the Medical Decision Extraction, Analysis, and Classification Task (MedExACT) at BioNLP 2026 addresses challenges in extracting medical decisions from lengthy ICU discharge summaries. Facing long document lengths and severe class imbalance within a limited 350-note training set, the team implemented a two-pronged strategy. First, a tripartite data augmentation pipeline, combining rule-based entity replacement, LLM-based contextual paraphrasing, and synthetic note generation, expanded the training data to over 2,300 notes. Second, they fine-tuned a domain-specific Clinical Longformer model, incorporating a sliding-window inference mechanism to handle sequences up to 2,048 tokens and Focal Loss to prioritize rare decision categories. This system, paired with a targeted post-processing module, achieved a Final Score of 0.5251 and a Token F1 of 0.6311, demonstrating robust performance across patient demographics.

Key takeaway

For NLP Engineers developing clinical text extraction systems, you should consider integrating a multi-stage data augmentation pipeline to overcome limited training data, especially for specialized tasks like medical decision extraction. Your approach should also prioritize domain-specific large language models, such as Clinical Longformer, combined with techniques like sliding-window inference and Focal Loss to effectively manage long document contexts and severe class imbalances, improving both detection accuracy and system stability.

Key insights

Combining data augmentation with domain-specific Longformer fine-tuning effectively extracts medical decisions from long, imbalanced clinical texts.

Principles

• Address data scarcity with multi-faceted augmentation.
• Use specialized models for domain-specific long sequences.
• Mitigate class imbalance with targeted loss functions.

Method

A two-pronged approach: first, expand training data from 350 to over 2,300 notes via rule-based, LLM-based, and synthetic augmentation; second, fine-tune Clinical Longformer with sliding-window inference and Focal Loss.

In practice

• Apply tripartite augmentation for scarce clinical data.
• Deploy Clinical Longformer for long medical documents.
• Implement Focal Loss for rare medical decision classes.

Topics

• Medical Decision Extraction
• Clinical Longformer
• Data Augmentation
• Natural Language Processing
• Class Imbalance
• BioNLP 2026

Best for: AI Scientist, NLP Engineer, Research Scientist

Related on AIssential

• Predicting relations between SOAP note sections: The value of incorporating a clinical information model — Fine-tuning a RoBERTa-base model on manually annotated clinical data improves NER for SOAP note sections.
• Cross-Lingual Empirical Evaluation of Large Language Models for Arabic Medical Tasks — English-centric LLMs exhibit significant performance gaps and reliability issues in Arabic medical tasks.
• Exploring automatic terminology extraction from historical medical data — Combining multiple automatic methods significantly improves terminology extraction from historical texts with non-standard orthography.
• EDEN: A Large-Scale Corpus of Clinical Notes for Italian — EDEN provides the largest Italian clinical note corpus and a CRF-filling benchmark for LLM-based information extraction.
• Measuring the sensitivity of LLM-based structured extraction to prompt, model, and schema choices in clinical discharge summaries — LLM-based clinical data extraction sensitivity varies significantly with model and schema, especially regarding "absence-versus-silence" distinctions.
• Comparing LLM and Fine-Tuned Model Performance on NVDRS Circumstance Extraction with Varying Prompt Complexity — LLMs, guided by a context-aware "Complexity Score" for prompt selection, significantly outperform fine-tuned models on inferentially complex, low-prevalence data extraction.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.