The Counterfactuals at SemEval-2026 Task 9: Can Counterfactually-Inspired Preprocessing help Detect Polarization?

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Advanced, medium

Summary

Teagan Johnson's "The Counterfactuals" team submitted English-language models for SemEval 2026 Task 9, which focuses on detecting multicultural online polarization. The task involves three subtasks: identifying if a post is polarized (subtask 1), classifying the polarization type (subtask 2), and identifying its manifestation (subtask 3). Johnson developed a pointwise mutual information-based lexicon to pinpoint words highly correlated with polarization, as labeled in subtask 1. This lexicon facilitated a large language model data augmentation technique. Subsequently, a BERTweet model was finetuned on the preprocessed datasets for each subtask. The models achieved rankings of 48th out of 60 for subtask 1, 35th out of 36 for subtask 2, and 17th out of 24 for subtask 3. All associated code is publicly available on GitHub.

Key takeaway

For NLP Engineers developing models for online polarization detection, consider integrating counterfactually-inspired preprocessing. Your approach could involve building a pointwise mutual information lexicon from initial polarized data, then using it to guide large language model data augmentation. This technique, while not top-performing in SemEval 2026 Task 9, offers a structured method to enhance dataset quality and potentially improve BERT-based model finetuning for nuanced social media analysis tasks.

Key insights

Counterfactually-inspired preprocessing and LLM data augmentation can be applied to detect online polarization.

Principles

• Pointwise mutual information can identify polarized lexicon.
• Data augmentation improves model performance.
• Finetuning BERT models is effective for text classification.

Method

Construct a pointwise mutual information-based lexicon from polarized data. Use this lexicon for large language model data augmentation. Finetune a BERT model (BERTweet) on the preprocessed datasets for specific classification subtasks.

In practice

• Use PMI to build domain-specific lexicons.
• Apply LLM data augmentation for text tasks.
• Finetune BERTweet for social media analysis.

Topics

• SemEval 2026
• Online Polarization Detection
• Pointwise Mutual Information
• LLM Data Augmentation
• BERTweet Finetuning
• Text Classification

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• mdok-style at SemEval-2026 Task 9: Finetuning LLMs for Multilingual Polarization Detection — Finetuning mid-sized LLMs with QLoRA and augmented data improves multilingual online polarization detection.
• CoCoGEC: Counterfactual Generation for Robust Grammatical Error Correction — Training GEC models with context-altered counterfactuals significantly enhances their robustness against contextual perturbations.
• From Text to Numbers: Demystifying Text Preprocessing in NLP — NLP text preprocessing, using methods like Bag of Words and TF-IDF, translates human language into numerical data for machine learning models.
• Graph-Augmented LLMs for Swiss MP Ideology Prediction — Graph-augmented LLMs can model complex political ideologies from text and relational data.
• Fully Automated Identification of Lexical Alignment and Preference-Stage Shifts in Large Language Models — Automated metrics can identify and quantify lexical overuse and preference learning shifts in LLMs without manual curation.
• IHUBERT: Vector-Based Semantic Deduplication and Domain-Balanced Pretraining for Persian Resources — Semantic deduplication and domain-balanced pretraining significantly enhance Persian PLM performance across diverse NLU tasks.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.