Creative Catalysts at #SMM4H-HeaRD 2026: XLM-RoBERTa for Task 1 Binary Classification of Social Media Posts Containing Adverse Drug Events

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Health & Wellbeing — Medical Devices & Health Technology, Pharmaceuticals & Biotechnology · Depth: Advanced, medium

Summary

Creative Catalysts developed a system for Task 1 of the Social Media Mining for Health Research and Applications Workshop (SMM4H 2026) to automatically detect Adverse Drug Events (ADEs) in social media posts. The team fine-tuned XLM-RoBERTa, a pre-trained model selected for its ability to handle multilingual content and linguistic diversity common in user-generated text. To address significant class imbalance, a class-weighting strategy was implemented, focusing the model on the underrepresented positive class. This adjustment led to an improved validation F1-score of 65%. The results underscore the effectiveness of transformer-based architectures for ADE detection and highlight the necessity of robust class-balancing techniques and multilingual generalization when processing real-world, imbalanced social media data.

Key takeaway

For Machine Learning Engineers developing healthcare NLP solutions, you should prioritize transformer-based models like XLM-RoBERTa for Adverse Drug Event detection from social media. When facing imbalanced datasets, integrate class-weighting strategies to improve performance on critical positive classes, as this approach yielded a 65% F1-score. This ensures your models are robust across diverse, multilingual user-generated content, enhancing real-world applicability and patient safety monitoring.

Key insights

Fine-tuned XLM-RoBERTa with class-weighting effectively detects Adverse Drug Events in multilingual, imbalanced social media data.

Principles

• Transformer architectures are effective for ADE detection.
• Class-balancing is critical for imbalanced datasets.
• Multilingual models generalize better on diverse social media.

Method

Fine-tune XLM-RoBERTa for binary classification, then apply a class-weighting strategy to prioritize the underrepresented positive class.

In practice

• Use XLM-RoBERTa for multilingual text classification.
• Implement class weighting for imbalanced datasets.

Topics

• Adverse Drug Events
• Social Media Mining
• XLM-RoBERTa
• Binary Classification
• Class Imbalance
• Natural Language Processing

Best for: NLP Engineer, AI Scientist, Machine Learning Engineer, Research Scientist

Related on AIssential

• From Logistic Regression to GPT-2: Building a Complete Spam Detection & Sentiment Analysis Pipeline — For imbalanced text classification, prioritize F1-score and Precision-Recall AUC over accuracy and ROC-AUC.
• Predicting relations between SOAP note sections: The value of incorporating a clinical information model — Fine-tuning a RoBERTa-base model on manually annotated clinical data improves NER for SOAP note sections.
• LinguIUTics at PsyDefDetect: Iterative Imbalance-Aware Fine-tuning of Qwen3-8B for Psychological Defense Mechanism Classification — Iterative imbalance-aware fine-tuning of LLMs, combined with augmentation and post-processing, significantly improves rare-class NLP performance.
• Holy $#!t: Are popular toxicity models simply profanity detectors? — Toxicity models often misclassify positive profanity due to inadequate training data and contextual understanding.
• MentalMARBERT: Domain-Adaptive Pre-training and Two-Stage Fine-Tuning for Arabic Mental Health Disorders Detection — Domain-adaptive pre-training and hierarchical classification significantly improve Arabic mental health disorder detection.
• mdok-style at SemEval-2026 Task 9: Finetuning LLMs for Multilingual Polarization Detection — Finetuning mid-sized LLMs with QLoRA and augmented data improves multilingual online polarization detection.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.