AI@UMS at SemEval-2026 Task 6: Handling Long Question-Answer Pairs with Sliding Window Models for Clarity and Evasion Analysis

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Natural Language Processing · Depth: Expert, quick

Summary

The AI@UMS system was developed for SemEval-2026 Task 6, named CLARITY - Unmasking Political Question Evasions. This system classifies question-answer (QA) pairs from political interviews based on clarity level (Subtask 1) and evasion technique (Subtask 2). A primary challenge involved handling long interview transcripts that often exceed the 512-token input limit of standard transformer encoder models. To overcome this, AI@UMS implemented a sliding-window fine-tuning strategy on roberta-base, segmenting QA pairs into overlapping 512-token windows with a 256-token stride. Predictions from individual windows were aggregated using softmax probability averaging across windows and an ensemble of three models trained with different random seeds. The system also employed class-weighted focal-inspired loss and label smoothing to address significant class imbalance. It achieved macro F1 scores of 0.62 for Subtask 1 and 0.48 for Subtask 2 on the official evaluation set.

Key takeaway

For NLP Engineers building classification systems with long text inputs, this work demonstrates effective strategies to overcome transformer token limits. You should consider implementing a sliding-window approach with overlapping segments, like the 512-token window and 256-token stride used here, especially when fine-tuning models such as roberta-base. Additionally, integrating ensemble methods and specialized loss functions, such as class-weighted focal-inspired loss and label smoothing, can significantly improve performance on datasets with pronounced class imbalance.

Key insights

AI@UMS uses sliding windows and ensemble methods with roberta-base to classify long political QA pairs for clarity and evasion, addressing token limits and class imbalance.

Principles

• Transformer input limits necessitate segmentation strategies.
• Ensemble modeling enhances prediction robustness.
• Class imbalance requires weighted loss and label smoothing.

Method

Fine-tune roberta-base with a sliding window (512 tokens, 256 stride) on QA pairs. Aggregate per-window softmax probabilities across windows and a three-model ensemble. Apply class-weighted focal-inspired loss and label smoothing.

In practice

• Segment long texts for transformer processing.
• Average softmax probabilities for window aggregation.
• Use focal-inspired loss for imbalanced classes.

Topics

• SemEval-2026 Task 6
• Political Evasion Analysis
• Sliding Window Models
• Transformer Fine-tuning
• Ensemble Learning
• Class Imbalance Mitigation

Best for: AI Engineer, Machine Learning Engineer, Research Scientist, AI Scientist, NLP Engineer

Related on AIssential

• Training and Finetuning Multimodal Embedding & Reranker Models with Sentence Transformers — Finetuning multimodal embedding models on domain-specific data significantly boosts performance over general-purpose models.
• Taming Voice Complexity with Dynamic Ensembles at Modulate — Ensemble Listening Models (ELMs) use dynamically routed, specialized small models for cost-effective, accurate Voice AI.
• ttda704 at SemEval-2026 Task 6: Structured Chain-of-Thought Prompting for Political Evasion Detection — Structured Chain-of-Thought prompting significantly improves political evasion detection over fine-tuning large language models.
• Rethinking the Role of Positional Encoding: Sliding-Window Transformers without PE Remain Turing Complete — Sliding-window transformers can achieve Turing completeness without explicit positional encodings due to inherent window-induced asymmetry.
• Use of Machine Learning techniques and Large Language Models for automatic evaluation of Celpe-Bras exam texts — BERT adaptations achieved the best automatic evaluation for Celpe-Bras texts, but at a higher computational cost.
• SURGELLM: Rethinking Multi-Task Evaluation through Task-Aware Feature Gating with Class-Balanced Normalization — SURGELLM enhances multi-task NLP performance by integrating task-aware feature gating and class-balanced normalization within a unified transformer framework.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.