Team TIET at #SMM4H-HeaRD 2026: Fine-tuned Biomedical Transformers with Language-Balanced Sampling for Patient Metadata and Multilingual ADE Detection

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Science & Research — Health & Medical Research, Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Expert, quick

Summary

Team TIET presented its systems for two shared tasks at #SMM4H-HeaRD 2026. For Task 5, focused on detecting patient metadata in SARS-CoV-2 sequencing papers, their approach involved iterative LLM prompting followed by fine-tuning BiomedBERT-base. This system utilized weighted cross-entropy loss and probability threshold optimization, achieving an F1 score of 0.760, surpassing the competition mean of 0.729. For Task 1, which addressed multilingual adverse drug event detection across six languages and an unseen Farsi subset, Team TIET fine-tuned XLM-RoBERTa-base. Their method incorporated a combined language- and class-balanced sampling strategy with per-language threshold tuning, resulting in a macro F1 of 0.497 overall, or 0.608 when excluding the Farsi subset. The team also reported empirical findings, including BERT+LLM ensemble failure with bimodal probability distributions, the advantage of base models over large variants with limited data, and the significance of language-balanced gradient contribution in multilingual classification.

Key takeaway

For NLP Engineers developing biomedical text classification systems, consider these findings to optimize your model performance. When working with limited datasets, prioritize fine-tuning base model variants like BiomedBERT-base or XLM-RoBERTa-base over larger alternatives. Implement language- and class-balanced sampling strategies, especially for multilingual tasks, to ensure robust gradient contributions. Additionally, fine-tune probability thresholds per language or task to maximize F1 scores, as demonstrated by the 0.760 F1 for metadata detection and 0.608 for multilingual ADE.

Key insights

Fine-tuned biomedical Transformers with balanced sampling excel in patient metadata and multilingual adverse drug event detection.

Principles

• Base models outperform large variants on limited data.
• Language-balanced gradients are crucial for multilingual classification.
• Bimodal probability distributions can cause ensemble failure.

Method

Fine-tune BiomedBERT-base with weighted cross-entropy for metadata; fine-tune XLM-RoBERTa-base with language- and class-balanced sampling for multilingual ADE.

In practice

• Apply weighted cross-entropy for imbalanced biomedical text.
• Use language-balanced sampling for multilingual NLP tasks.
• Optimize probability thresholds per language/task.

Topics

• Biomedical NLP
• Adverse Drug Event Detection
• Transformer Models
• Fine-tuning
• Language-Balanced Sampling
• SARS-CoV-2 Metadata

Best for: AI Engineer, Research Scientist, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• LinguIUTics at PsyDefDetect: Iterative Imbalance-Aware Fine-tuning of Qwen3-8B for Psychological Defense Mechanism Classification — Iterative imbalance-aware fine-tuning of LLMs, combined with augmentation and post-processing, significantly improves rare-class NLP performance.
• Cross-Lingual Empirical Evaluation of Large Language Models for Arabic Medical Tasks — English-centric LLMs exhibit significant performance gaps and reliability issues in Arabic medical tasks.
• Predicting relations between SOAP note sections: The value of incorporating a clinical information model — Fine-tuning a RoBERTa-base model on manually annotated clinical data improves NER for SOAP note sections.
• Trade-offs in Medical LLM Adaptation: An Empirical Study in French QA — For medical LLM adaptation, SFT is cost-effective for MCQA, but CPT or CPT+SFT is preferred for OEQA, showing cross-lingual transfer.
• Severity-Aware Curriculum Learning with Multi-Model Response Selection for Medical Text Generation — Severity-aware curriculum learning with multi-model response selection significantly improves medical text generation quality and relevance.
• Comparing LLM and Fine-Tuned Model Performance on NVDRS Circumstance Extraction with Varying Prompt Complexity — LLMs, guided by a context-aware "Complexity Score" for prompt selection, significantly outperform fine-tuned models on inferentially complex, low-prevalence data extraction.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.