SMASH at SemEval-2026 Task 9: Detecting Multilingual Polarisation with Encoder Ensembles and Calibrated Decision Thresholds

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Expert, medium

Summary

The SMASH system was submitted to SemEval-2026 Task 9, focusing on multilingual, multicultural, and multi-event polarisation detection. This task involved three subtasks: binary polarisation detection, multi-label classification of polarisation types, and multi-label identification of polarisation manifestations across various languages. SMASH proposes a language-adaptive ensemble framework that combines monolingual and multilingual encoder-only transformers. A key component is a principled out-of-fold (OOF) threshold tuning strategy, which jointly optimizes ensemble weights and class-wise decision thresholds to maximize macro-F1 scores. Experiments revealed that monolingual encoders excel in high-resource languages but benefit from multilingual signals, and language-specific class threshold tuning significantly improves performance due to diverse cross-lingual class distributions. Across 22 evaluation languages, SMASH ranked among the top three systems for 5 languages in Subtask 1, 14 languages in Subtask 2, and 16 languages in Subtask 3, achieving average macro-F1 scores of 0.81, 0.62, and 0.53 for Subtasks 1, 2, and 3, respectively.

Key takeaway

For NLP Engineers developing multilingual classification systems, you should prioritize language-adaptive ensemble frameworks. Implementing a principled out-of-fold threshold tuning strategy, which jointly optimizes ensemble weights and class-wise decision thresholds, will significantly boost your model's performance on imbalanced, multi-label tasks. This approach is particularly effective for handling cross-lingual variations in class distributions, ensuring robust results across diverse languages and subtasks.

Key insights

Language-adaptive ensembles with calibrated decision thresholds significantly improve multilingual polarisation detection performance.

Principles

• Monolingual encoders benefit from complementary multilingual signals.
• No single multilingual backbone universally outperforms others.
• Language-specific class threshold tuning substantially improves performance.

Method

A language-adaptive ensemble framework combines monolingual and multilingual encoder-only transformers. It uses an out-of-fold (OOF) strategy to jointly tune ensemble weights and class-wise decision thresholds for macro-F1 optimization.

In practice

• Combine monolingual and multilingual models.
• Implement language-specific threshold tuning.
• Optimize directly for macro-F1.

Topics

• Multilingual Polarisation Detection
• Encoder Ensembles
• Transformer Models
• Threshold Tuning
• SemEval-2026
• Multi-label Classification

Best for: Research Scientist, AI Scientist, NLP Engineer, Machine Learning Engineer

Related on AIssential

• Multi-Stage Training for Abusive Comment Detection in Indic Languages — A multi-stage pipeline combining language preprocessing and model ensembles effectively detects abusive Indic language comments while minimizing false positives.
• CAMO: A Class-Aware Minority-Optimized Ensemble for Robust Language Model Evaluation on Imbalanced Data — CAMO is a hierarchical ensemble method that dynamically boosts minority class predictions using confidence and uncertainty.
• SURGELLM: Rethinking Multi-Task Evaluation through Task-Aware Feature Gating with Class-Balanced Normalization — SURGELLM enhances multi-task NLP performance by integrating task-aware feature gating and class-balanced normalization within a unified transformer framework.
• mdok-style at SemEval-2026 Task 9: Finetuning LLMs for Multilingual Polarization Detection — Finetuning mid-sized LLMs with QLoRA and augmented data improves multilingual online polarization detection.
• OpenLID-v3: Improving the Precision of Closely Related Language Identification -- An Experience Report — Improving language identification for closely related languages requires specialized benchmarks and handling of non-language content.
• TeamHerald@CHIPSAL 2026: Hate Speech Detection and Sentiment Analysis of Nepali Memes using Transformer-based Architectures and Ensemble Learning — Ensemble strategies for meme analysis perform differently for binary vs. multi-class tasks, requiring task-specific aggregation.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.