Otter at MedExAct2026: Diverse Encoder Ensemble for Medical Decision Span Detection

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Biomedical Natural Language Processing · Depth: Expert, short

Summary

Otter at MedExAct2026 presents an ensemble of 10 transformer encoders designed for the MedExACT 2026 shared task, focusing on medical decision span detection. This ensemble achieves diversification through three key training approaches: varied encoder initialization, which includes domain-adaptive pre-training on clinical text; distinct loss functions; and robust data augmentation. The data augmentation strategy incorporates both LLM-generated synthetic notes and silver-labeled clinical documents. A greedy forward search mechanism is employed to identify the optimal combination of these diversified components, aiming for the highest validation final score. Furthermore, a BERT-based boundary refiner is integrated to correct any offset errors in the ensemble's predicted spans prior to final submission.

Key takeaway

For NLP Engineers developing medical text analysis systems, you should consider implementing diverse transformer encoder ensembles. This approach, incorporating domain-adaptive pre-training and varied data augmentation techniques like LLM-generated notes, can significantly enhance the accuracy of medical decision span detection. Integrate a BERT-based boundary refiner into your pipeline to minimize offset errors, ensuring higher precision in clinical document processing.

Key insights

A diverse ensemble of 10 transformer encoders, optimized with domain-adaptive pre-training and data augmentation, improves medical decision span detection.

Principles

• Ensemble diversification enhances performance.
• Domain-adaptive pre-training is crucial for clinical NLP.
• Data augmentation improves model robustness.

Method

Build an ensemble of 10 transformer encoders, diversifying initialization, loss functions, and data augmentation (LLM-generated/silver-labeled). Select optimal combination via greedy forward search. Refine predicted spans with a BERT-based boundary corrector.

In practice

• Use diverse encoder ensembles for NLP tasks.
• Apply BERT-based refiners for span correction.
• Integrate LLM-generated data for augmentation.

Topics

• Medical Decision Span Detection
• Transformer Ensembles
• Domain-Adaptive Pre-training
• Data Augmentation
• BERT Boundary Refiner
• Clinical Natural Language Processing

Best for: AI Scientist, NLP Engineer, Research Scientist

Related on AIssential

• L2D-Clinical: Learning to Defer for Adaptive Model Selection in Clinical Text Classification — Adaptive model selection in clinical text classification improves accuracy by deferring to LLMs based on uncertainty.
• From TF-IDF to Transformers: A Comparative and Ensemble Approach to Sentiment Classification — A soft voting ensemble of diverse machine learning and transformer models significantly enhances sentiment classification accuracy, particularly for nuanced text.
• Domain Fine-Tuning FinBERT on Finnish Histopathological Reports: Train-Time Signals and Downstream Correlations — Domain fine-tuning FinBERT on medical text can be predicted by observing embedding geometry changes.
• Label-Aware Pseudo-Training Sample Generation for Text Classification — A new data augmentation method uses LLMs to subtly alter sentences, improving NLP performance with limited data.
• AI-assisted Protocol Information Extraction For Improved Accuracy and Efficiency in Clinical Trial Workflows — AI-assisted RAG significantly boosts clinical trial protocol data extraction accuracy and efficiency over standalone LLMs.
• Linguistic Symptoms: Augmented Generation by Retrieval and Reasoning in LLMs under Portuguese-English Variation in Medical Contexts — Linguistic variation significantly impacts smaller LLMs in medical contexts, with larger models showing greater cross-language robustness.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.