Gladiators at #SMM4H–HeaRD 2026: Multi-Seed XLM-RoBERTa Ensemble with Focal Loss and Per-Language Threshold Optimization for Multilingual Adverse Drug Event Detection

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics, Natural Language Processing · Depth: Expert, quick

Summary

The Gladiators system, developed for Task 1 of the SMM4H 2026 shared task, addresses the binary classification of adverse drug event (ADE) mentions within multilingual social media posts. This system fine-tunes three XLM-RoBERTa large models, each initialized with a different random seed, employing focal loss (α=0.75, γ=2.0) and 3× positive oversampling. After training, the system averages the models' predicted probabilities and applies per-language threshold optimization. On the development set, it achieved a pooled binary F1 score of 0.7505. During the official test set, which included a surprise Farsi language component comprising 35.5% of samples, the system attained an F1 score of 0.6039, surpassing the competition's mean (0.5465) and median (0.5798). The team also evaluated eleven different approaches, documenting key negative results, and later improved the development F1 to 0.7585 with a six-model cross-regime ensemble.

Key takeaway

For NLP Engineers developing multilingual classification systems, particularly for critical health-related tasks like adverse drug event detection, you should consider implementing multi-seed XLM-RoBERTa ensembles. This approach, combined with focal loss for imbalanced data and per-language threshold optimization, significantly improves performance and robustness, even when encountering unexpected languages like Farsi. Your systems will benefit from this strategy's ability to maintain high F1 scores across diverse linguistic inputs.

Key insights

A multi-seed XLM-RoBERTa ensemble with focal loss and per-language threshold optimization effectively detects adverse drug events in multilingual social media.

Principles

• Ensemble learning enhances model robustness.
• Per-language thresholding improves multilingual performance.
• Focal loss and oversampling mitigate class imbalance.

Method

Fine-tune three XLM-RoBERTa large models with different seeds, focal loss (α=0.75, γ=2.0), and 3× positive oversampling, then average probabilities and optimize thresholds per language.

In practice

• Implement multi-seed ensembles for stability.
• Apply focal loss for imbalanced binary classification.
• Optimize prediction thresholds for each target language.

Topics

• Adverse Drug Event Detection
• Multilingual NLP
• XLM-RoBERTa
• Ensemble Learning
• Focal Loss
• Social Media Mining

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• mdok-style at SemEval-2026 Task 9: Finetuning LLMs for Multilingual Polarization Detection — Finetuning mid-sized LLMs with QLoRA and augmented data improves multilingual online polarization detection.
• Ensembles of Large Language Models for Identifying EQ-5D Studies in PubMed Based on Their Abstracts — Ensemble LLM frameworks reliably automate biomedical text classification, balancing performance and interpretability.
• Multi-Stage Training for Abusive Comment Detection in Indic Languages — A multi-stage pipeline combining language preprocessing and model ensembles effectively detects abusive Indic language comments while minimizing false positives.
• Automated Detection of Dosing Errors in Clinical Trial Narratives: A Multi-Modal Feature Engineering Approach with LightGBM — Combining sparse lexical features with dense semantic embeddings improves clinical dosing error detection.
• Lost in Quantization: Activation Outliers Explain Language-Specific FP8 Sensitivity in Llama-3 — Activation outliers explain language-specific FP8 quantization sensitivity in LLMs like Llama-3.
• Multilingual Hematology Visual Question Answering Dataset — Multilingual VQA datasets are crucial for equitable medical AI, bridging language gaps in healthcare.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.