BioSentinel at EXIST 2026: Soft-Label Optimization with XLM-RoBERTa for Sexism Intent Classification in Memes

2026-07-27 · Source: Computation and Language · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Natural Language Processing · Depth: Expert, quick

Summary

The BioSentinel team participated in EXIST 2026 Task 2.2, "Source Intention in Memes," as part of the CLEF 2026 evaluation campaign. Their system classifies the communicative intent of memes into direct, judgemental, or non-sexist categories, adhering to a Learning with Disagreement (Le-Wi-Di) paradigm that requires both hard-label and soft-label predictions. The approach is text-centric, utilizing an xlm-roberta-base model with 270 million parameters. Training involved a composite loss function combining KL divergence for soft annotator distributions and weighted cross-entropy for hard labels. On the official test set, the system achieved an ICM-Soft-Norm of 0.3229 and an ICM-Norm of 0.3778, alongside a hard F1-score of 0.4236. These results placed BioSentinel 40th out of 118 submissions in soft-soft evaluation and 49th out of 187 in hard-hard evaluation. The team also analyzed dataset characteristics, explored larger architectures, and examined annotator disagreement's impact on subjective NLP tasks, noting that KL loss improved soft-label metrics while cross-entropy loss enhanced hard-label accuracy.

Key takeaway

For NLP Engineers developing models for subjective classification tasks like intent detection in memes, you should consider implementing a composite loss function. Combining KL divergence for soft-label distributions with weighted cross-entropy for hard labels can significantly improve performance across both soft and hard evaluation metrics. This approach is particularly valuable when dealing with annotator disagreement, allowing your models to capture nuanced human judgments more effectively and achieve better rankings in competitive evaluations.

Key insights

Combining KL divergence and cross-entropy loss effectively addresses annotator disagreement in subjective meme intent classification.

Principles

• Subjective NLP benefits from soft-label predictions.
• KL loss optimizes soft-label metrics.
• CE loss optimizes hard-label accuracy.

Method

Train xlm-roberta-base with a composite loss function: KL divergence for soft annotator distributions and weighted cross-entropy for hard labels, to classify meme intent under a Le-Wi-Di paradigm.

In practice

• Classify sexism intent in memes.
• Apply Le-Wi-Di for subjective NLP tasks.
• Optimize models for soft-label outputs.

Topics

• Sexism Classification
• Meme Analysis
• XLM-RoBERTa
• Soft-Label Optimization
• Learning with Disagreement
• Composite Loss Functions

Best for: Research Scientist, AI Scientist, NLP Engineer

Related on AIssential

• AlphaLyrae at SemEval-2026 Task 9: Metric Learning and Asymmetric Loss for Chinese Polarization Analysis — Metric learning and asymmetric loss effectively address implicit language and class imbalance in online polarization detection.
• Metric-Dependent Annotation Saturation for Learning from Label Distributions — Annotation budget needs depend on the target evaluation metric, as different metrics saturate at varying annotator counts.
• Human Label Variation as Stable Signal: Learning Annotator-Specific Explanation Behavior via Cross-Annotator Preference Optimization — Large language models can learn and reproduce individual annotator explanation styles from human label variation.
• Symmetrization of Loss Functions for Robust Training of Neural Networks in the Presence of Noisy Labels — Symmetrizing multi-class loss functions, especially cross-entropy, yields a unique convex unhinged loss robust to noisy labels.
• Consistently Informative Soft-Label Temperature for Knowledge Distillation — Fixed-temperature knowledge distillation creates inconsistent soft-label entropy, hindering effective knowledge transfer.
• Plug-in Losses for Evidential Deep Learning: A Simplified Framework for Uncertainty Estimation that Includes the Softmax Classifier — A simplified Evidential Deep Learning framework uses plug-in losses to achieve reliable, efficient uncertainty estimation, integrating softmax for broader applicability.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computation and Language.