BertKittens at SemEval-2026 Task 3: Multi-Domain Aspect Sentiment with BERT/DeBERTa Ensembles for VA Regression and Aspect–Opinion–VA Triplets

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Expert, quick

Summary

The BertKittens system, presented at SemEval-2026 Task 3, addresses multi-domain aspect sentiment analysis, specifically Valence–Arousal (VA) regression and aspect–opinion–VA triplet extraction. Built on BERT and DeBERTa transformer encoders, the system employs a multi-task learning framework. For the regression subtask, it jointly predicts VA scores and token-level opinion spans using a shared encoder and task-specific output heads. This joint formulation introduces auxiliary token-level supervision, which stabilizes training and enhances regression accuracy compared to single-task optimization. While achieving strong performance with provided gold annotations, the system's accuracy substantially degrades in fully end-to-end scenarios due to error propagation from automatic token-level predictions.

Key takeaway

For NLP engineers developing multi-domain aspect sentiment systems, consider implementing multi-task learning with auxiliary token-level supervision for Valence–Arousal regression. This approach can stabilize training and improve accuracy, but be aware that end-to-end performance may degrade significantly if automatic span extraction is required, necessitating careful evaluation of error propagation in real-world deployments.

Key insights

Jointly predicting VA scores and opinion spans with auxiliary supervision improves sentiment regression accuracy.

Principles

• Auxiliary token-level supervision stabilizes multi-task training.
• Joint affective regression and span modeling offers benefits.
• End-to-end evaluation exposes transformer sequence labeling limits.

Method

The system uses BERT/DeBERTa transformer encoders fine-tuned in a multi-task learning framework, employing a shared encoder with task-specific output heads for joint VA regression and token-level opinion span prediction.

In practice

• Apply multi-task learning for improved sentiment regression.
• Consider auxiliary token-level tasks to stabilize training.
• Evaluate end-to-end performance carefully for span extraction.

Topics

• Aspect Sentiment Analysis
• Valence-Arousal Regression
• Multi-task Learning
• BERT
• DeBERTa
• Transformer Encoders
• SemEval-2026

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• Sentiment Analysis with BERT: The Heart of the Transformer, The Soul of the Text — BERT's bidirectional context understanding via Transformers significantly advanced sentiment analysis and NLP tasks.
• AspectRAG: An Architecture of Retrieval and Generation for Aspect-Based Sentiment Analysis — AspectRAG leverages aspect-guided retrieval and generation for unsupervised, high-performance sentiment analysis in Portuguese.
• Beyond Sentiment Classification: A Generative Framework for Emotion Intensity Evaluation in Text — Generative LLMs fine-tuned on continuous emotion intensity scores outperform classification and generalize to unseen affective dimensions.
• Structured Sentiment Analysis in Brazilian Portuguese: An Exploratory Study Using BERTimbau — Span-level sentiment extraction provides a viable baseline for Structured Sentiment Analysis in Brazilian Portuguese.
• Teaching Machines to “Feel” Tweets — Domain-native pre-trained models are crucial for accurate sentiment classification in ambiguous social media language.
• DiffuSent: Towards a Unified Diffusion Framework for Aspect-Based Sentiment Analysis — DiffuSent unifies ABSA subtasks via non-auto-regressive boundary denoising, improving accuracy and speed for multi-word sentiment extraction.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.