Duluth at SemEval-2026 Task 6: DeBERTa with LLM-Augmented Data for Unmasking Political Question Evasions

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Expert, medium

Summary

The Duluth approach to SemEval-2026 Task 6 addresses CLARITY: Unmasking Political Question Evasions, specifically Task 1 (clarity-level classification) and Task 2 (evasion-level classification). This system classifies question-answer pairs from U.S. presidential interviews based on a two-level response clarity taxonomy. It is built upon DeBERTa-V3-base, augmented with focal loss, layer-wise learning rate decay, and boolean discourse features. To mitigate class imbalance in the training data, the team generated synthetic examples for minority classes using Gemini 3 and Claude Sonnet 4.5. The best configuration achieved a Macro F1 of 0.76 on the Task 1 evaluation set, securing 8th place among 40 teams. This score compares to the top-ranked TeleAI system's 0.89 and the participant mean of 0.70. Error analysis highlighted confusion between "Ambivalent" and "Clear Reply" responses, mirroring human annotator disagreements. The findings demonstrate that LLM-based data augmentation effectively enhances minority-class recall in nuanced political discourse tasks.

Key takeaway

For NLP engineers developing classification systems for nuanced text, consider integrating LLM-based data augmentation to address class imbalance. Your models, especially on tasks like political discourse analysis, can achieve better minority-class recall by generating synthetic examples with tools like Gemini 3 or Claude Sonnet 4.5. This approach can significantly improve performance on challenging categories, even when human annotation shows similar ambiguities.

Key insights

LLM-augmented data significantly improves minority-class recall in political question evasion detection.

Principles

• Class imbalance impacts nuanced classification.
• LLM-generated data can balance datasets.
• Human annotation disagreement reflects model challenges.

Method

The Duluth system uses DeBERTa-V3-base, enhanced with focal loss, layer-wise learning rate decay, and boolean discourse features. It augments minority classes with synthetic data from Gemini 3 and Claude Sonnet 4.5.

In practice

• Apply focal loss for imbalanced classification.
• Use LLMs like Gemini 3 for data augmentation.
• Integrate discourse features into NLP models.

Topics

• SemEval-2026 Task 6
• DeBERTa-V3-base
• LLM Data Augmentation
• Political Question Evasion
• Class Imbalance
• Gemini 3
• Claude Sonnet 4.5

Best for: AI Engineer, Research Scientist, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• Label-Aware Pseudo-Training Sample Generation for Text Classification — A new data augmentation method uses LLMs to subtly alter sentences, improving NLP performance with limited data.
• Graph-Augmented LLMs for Swiss MP Ideology Prediction — Graph-augmented LLMs can model complex political ideologies from text and relational data.
• mdok-style at SemEval-2026 Task 10: Finetuning LLMs for Conspiracy Detection — Data augmentation and self-training can effectively finetune LLMs for specialized text classification tasks with limited data.
• Enhancing Science Classroom Discourse Analysis through Joint Multi-Task Learning for Reasoning-Component Classification — Automated discourse analysis using LLM augmentation improves utterance classification and reveals effective teaching patterns.
• Addressing Data Scarcity in Bangla Fake News Detection: An LLM-Based Dataset Augmentation Approach — LLM-based data augmentation significantly improves fake news detection in low-resource languages by generating diverse synthetic data.
• A Systematic Analysis of Biases in Large Language Models — LLMs, despite alignment efforts, inherit and perpetuate diverse biases from human-generated training data.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.