SG-UniBuc-NLP at SemEval-2026 Task 6: Multi-Head RoBERTa with Chunking for Long-Context Evasion Detection

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Expert, quick

Summary

SG-UniBuc-NLP developed a system for SemEval-2026 Task 6 (CLARITY: Unmasking Political Question Evasions), designed to classify English political interview responses. The system performs both coarse-grained clarity classification (3-way) and fine-grained evasion strategy classification (9-way). To overcome the 512-token limit of standard Transformer encoders for long responses, it employs an overlapping sliding-window chunking strategy with element-wise Max-Pooling aggregation. A shared RoBERTa-large encoder powers two task-specific heads, trained jointly using a multi-task objective. Inference-time ensembling is applied over 7-fold stratified cross-validation. The system achieved a Macro-F1 of 0.80 on Subtask 1 and 0.51 on Subtask 2, securing 11th place in both.

Key takeaway

For Machine Learning Engineers building long-context text classification systems, this approach offers a robust strategy. If your inputs frequently exceed standard Transformer token limits, you should consider implementing overlapping sliding-window chunking with Max-Pooling. Combining this with a shared encoder and multi-task learning for related classification objectives can significantly improve performance and model efficiency, as demonstrated by the 0.80 Macro-F1 on clarity detection.

Key insights

A multi-head RoBERTa system uses chunking and Max-Pooling to classify political question evasion in long contexts.

Principles

• Chunking handles long text sequences for Transformers.
• Multi-task learning improves related classification tasks.
• Ensembling boosts model robustness and performance.

Method

The system applies overlapping sliding-window chunking with element-wise Max-Pooling over chunk representations from a shared RoBERTa-large encoder, feeding two jointly trained task-specific heads.

In practice

• Implement overlapping chunking for long text inputs.
• Use multi-task learning for related NLP classification.
• Apply cross-validation ensembling for robust models.

Topics

• SemEval-2026
• Evasion Detection
• Long-Context NLP
• RoBERTa
• Multi-task Learning
• Text Chunking

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• Toward General Semantic Chunking: A Discriminative Framework for Ultra-Long Documents — A discriminative Qwen3-0.6B model significantly improves ultra-long document topic segmentation speed and accuracy over generative LLMs.
• An End-to-End Hybrid Framework for Rumour Detection in Low-Resources Algerian Dialect — A hybrid framework combining transformer embeddings with classical classifiers enables effective rumour detection in low-resource Algerian dialect.
• A Beginner’s Guide to NLP Token Classification: NER, POS Tagging, Chunking, and BERT — Token classification, including NER, POS tagging, and chunking, is fundamental for machines to understand human language.
• Attention Expansion: Enhancing Keyphrase Extraction from Long Documents with Attention-Augmented Contextualized Embeddings — Attention expansion efficiently broadens PLM context for keyphrase extraction by integrating lightweight out-of-context word embeddings.
• mdok-style at SemEval-2026 Task 9: Finetuning LLMs for Multilingual Polarization Detection — Finetuning mid-sized LLMs with QLoRA and augmented data improves multilingual online polarization detection.
• SkMTEB: Slovak Massive Text Embedding Benchmark and Model Adaptation — SkMTEB provides a robust benchmark and efficient, language-specific embedding models for Slovak, a low-resource language.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.