Limics at #SMM4H-HeaRD 2026: Uncertainty-Driven Prediction for ADE Detection in Social Media

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics, Health & Medical Research · Depth: Advanced, short

Summary

Limics developed a two-stage pipeline for the SMM4H-HeaRD 2026 Task 1, focusing on detecting Adverse Drug Events (ADE) in multilingual, multi-platform social media posts. The system combines a fine-tuned XLM-RoBERTa-large encoder-only model with a large language model, which handles final decisions for ambiguous cases. To address complex linguistic boundaries, the encoder was explicitly trained to recognize ambiguity as a discrete third label, delegating these instances to the generative model. While this third label approach initially showed lower performance than a binary model, using the encoder as a preliminary filter, classifying 78.62% of posts as negatives, resulted in a global F1 score of 0.614. This score represents a +0.034 improvement over the task median, demonstrating an effective strategy for ADE detection.

Key takeaway

For NLP Engineers building adverse event detection systems, consider a two-stage classification pipeline. Train your initial encoder model to explicitly flag ambiguous cases, delegating them to a more capable large language model. This approach, achieving a 0.614 F1 score (+0.034 over median) for ADE detection, efficiently filters clear negatives. It dedicates advanced processing to complex, uncertain instances, improving overall accuracy and reliability.

Key insights

A two-stage pipeline combining an encoder for initial classification and a large language model for ambiguous cases improves ADE detection.

Principles

• Delegate ambiguous cases to a more powerful generative model.
• Train initial classifiers to explicitly identify ambiguous inputs.

Method

A two-stage pipeline first uses a fine-tuned XLM-RoBERTa-large encoder to classify posts, including an "ambiguous" label. A large language model then resolves these ambiguous cases, after the encoder filters 78.62% as negatives.

In practice

• Design multi-stage NLP systems for complex classification tasks.
• Incorporate an "uncertainty" class in initial model training.
• Filter clear negatives early with a high-recall encoder.

Topics

• Adverse Drug Events
• Social Media Mining
• XLM-RoBERTa-large
• Large Language Models
• Ambiguity Detection
• Two-stage Classification

Best for: AI Scientist, NLP Engineer, Research Scientist

Related on AIssential

• Multi-Stage Training for Abusive Comment Detection in Indic Languages — A multi-stage pipeline combining language preprocessing and model ensembles effectively detects abusive Indic language comments while minimizing false positives.
• Neural Bayesian Anomaly Mitigation: A Robust Loss that Doubles as an Unsupervised Contamination Classifier — NBAM is a robust loss that simultaneously classifies contamination using a Bayesian latent-switch mixture model.
• From Hazard Functions to Language Space: Cox-Supervised Distillation of Survival Risk into a Large Language Model — Large language models can internalize and predict survival risk from structured clinical data via text-based distillation.
• Noise reduction in BERT NER models for clinical entity extraction — A post-processing Noise Removal model significantly boosts clinical NER precision by analyzing probability distributions and contextual features.
• AI scans 400,000 Reddit posts and finds hidden Ozempic side effects — AI-driven social media analysis can rapidly identify underreported drug side effects missed by traditional clinical trials.
• A Reasoner for Real-World Event Detection: Scaling Reinforcement Learning via Adaptive Perplexity-Aware Sampling Strategy — APARL uses perplexity-aware reinforcement learning and dynamic curriculum to improve abnormal event detection and OOD generalization.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.