LingoResearchGroup at SemEval-2026 Task 9: Evaluating Prompt Variants for Polarization Detection

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Expert, medium

Summary

LingoResearchGroup submitted a system to SemEval-2026 Task 9, the Multilingual Text Classification Challenge for Polarization Detection, addressing three subtasks: binary polarization detection, polarization type classification, and polarization manifestation identification. Their approach systematically evaluated twelve distinct prompts, varying in terminology clarity, definition detail, reasoning guidance, and in-context examples. Experiments utilized aya-101 and Gemma3-27B, with Gemma3-27B chosen for the final submission due to performance. The system achieved an average macro F1-score of 0.762 for Subtask 1, 0.587 for Subtask 2, and 0.444 for Subtask 3, alongside average accuracies of 0.819, 0.678, and 0.498, respectively, across 22 languages. Analysis indicated prompt-based methods effectively detect coarse-grained polarization but face increasing difficulty with fine-grained and multi-label sociolinguistic classification.

Key takeaway

For NLP Engineers developing multilingual polarization detection systems, you should prioritize prompt engineering, systematically testing variants that differ in clarity, definition, reasoning guidance, and in-context examples. While prompt-based LLMs like Gemma3-27B are effective for coarse-grained binary polarization, recognize their current limitations for fine-grained or multi-label sociolinguistic classification. Focus on refining prompts for simpler tasks before tackling more complex, nuanced detection challenges.

Key insights

Prompt-based LLMs effectively detect coarse-grained polarization but struggle with fine-grained sociolinguistic classification.

Principles

• Prompt design impacts polarization detection performance.
• Coarse-grained polarization is more detectable via prompts.
• Fine-grained sociolinguistic classification remains challenging.

Method

Systematically evaluate prompt variants differing in clarity, definition, reasoning guidance, and in-context examples using LLMs like Gemma3-27B for multilingual text classification.

In practice

• Test diverse prompt structures for classification.
• Apply prompt-based LLMs for binary polarization.
• Recognize limitations for complex multi-label tasks.

Topics

• Polarization Detection
• Prompt Engineering
• Multilingual Text Classification
• Large Language Models
• SemEval-2026
• Gemma3-27B

Best for: AI Engineer, Machine Learning Engineer, Research Scientist, AI Scientist, NLP Engineer, Prompt Engineer

Related on AIssential

• mdok-style at SemEval-2026 Task 9: Finetuning LLMs for Multilingual Polarization Detection — Finetuning mid-sized LLMs with QLoRA and augmented data improves multilingual online polarization detection.
• Supervision versus Demonstration-Based In-Context Learning for Multiword Expression Classification — Prompt sensitivity and demonstration quality critically impact LLM performance in metalinguistic classification tasks.
• A Multitask Transformer for Offensive Language Detection and Target Identification in HateBR — Multitask learning improves offensive language detection consistency but may reduce fine-grained target attribution.
• Linguistic Issues in Computational Linguistics for Non-English Languages — English-centric LLM design creates systematic linguistic weaknesses for non-English languages across multiple levels.
• As Easy as Rocket Science: Assessing the Ability of Large Language Models to Interpret Negation in Figurative Language — LLMs struggle with combined negation and figurative language, with prompt style significantly impacting performance.
• Muted: Multilingual Targeted Offensive Speech Identification and Visualization — Muted identifies targeted offensive speech using transformer models and spaCy, requiring no fine-tuning for toxic span detection.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.