Phatthachdau at SemEval-2026 Task 9: A Multi-Stage Augment-Judge-Train Pipeline for Multilingual Online Polarization Detection

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Expert, short

Summary

Phatthachdau's submission to SemEval-2026 Task 9 introduced a Multi-Stage Augment-Judge-Train (AJT) pipeline to tackle extreme label imbalance in the Hausa dataset, where only 11% of instances were polarized. The pipeline utilized Gemini 2.0 for taxonomy-driven data generation and an LLM-as-a-Judge layer for quality control, successfully expanding the minority class sixfold. An ensemble architecture, combining specialized Encoders with LLM-LORA, achieved 1st Place in Hausa with a 0.8336 Macro-F1 score and ranked in the Top 10 for English. These results underscore the effectiveness of culture-aware synthetic data in enhancing social NLP applications for low-resource languages.

Key takeaway

For NLP Engineers developing solutions for low-resource languages, you should consider multi-stage data augmentation pipelines like the Augment-Judge-Train (AJT) approach. This method, which uses LLMs for taxonomy-driven synthetic data generation and quality control, can significantly address extreme label imbalance and improve model performance. Implementing such a pipeline can lead to top-tier results in tasks like online polarization detection, even for languages with limited data.

Key insights

The Augment-Judge-Train (AJT) pipeline effectively addresses label imbalance in low-resource languages using LLM-driven synthetic data.

Principles

• Culture-aware synthetic data enhances social NLP.
• LLM-as-a-Judge improves data quality control.
• Ensemble architectures boost multilingual performance.

Method

The AJT pipeline involves using Gemini 2.0 for taxonomy-driven data generation, an LLM-as-a-Judge for quality control, and then training an ensemble of specialized Encoders with LLM-LORA.

In practice

• Generate synthetic data with LLMs like Gemini 2.0.
• Implement LLM-as-a-Judge for data quality.
• Combine Encoders with LLM-LORA for robust models.

Topics

• Multilingual NLP
• Online Polarization Detection
• Data Augmentation
• Low-Resource Languages
• Augment-Judge-Train Pipeline
• LLM-as-a-Judge
• Gemini 2.0

Best for: Research Scientist, AI Engineer, AI Scientist, NLP Engineer, Machine Learning Engineer

Related on AIssential

• mdok-style at SemEval-2026 Task 9: Finetuning LLMs for Multilingual Polarization Detection — Finetuning mid-sized LLMs with QLoRA and augmented data improves multilingual online polarization detection.
• Low-resource Language Discrimination Towards Chinese Dialects with Transfer learning and Data Augmentation — Transfer learning and data augmentation effectively overcome low-resource challenges in Chinese dialect discrimination.
• Multi-Stage Training for Abusive Comment Detection in Indic Languages — Multi-stage training with language preprocessing and model ensembles improves abusive comment detection while minimizing false positives.
• IndicDB -- Benchmarking Multilingual Text-to-SQL Capabilities in Indian Languages — IndicDB reveals a significant "Indic Gap" in LLM Text-to-SQL performance for Indian languages compared to English.
• TalkTag: Fine-Grained Morphosyntactic Error Annotation for Transcribed Speech — TalkTag automates fine-grained morphosyntactic error annotation in spoken transcripts using an LLM, even with scarce data.
• An End-to-End Hybrid Framework for Rumour Detection in Low-Resources Algerian Dialect — A hybrid framework combining transformer embeddings with classical classifiers enables effective rumour detection in low-resource Algerian dialect.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.