hdharpure at SemEval-2026 Task 3: BERT-Based Modeling and Prediction Behavior Analysis for Multilingual Valence–Arousal Scoring

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Expert, medium

Summary

Harshal Dharpure and Nicolay Rusnachenko presented a BERT-based model for SemEval-2026 Task 3, a Dimensional aspect-based sentiment analysis (DimABSA) challenge. This task, specifically Track A/Subtask 1, requires predicting continuous valence (V) and arousal (A) scores for aspects in multilingual texts. Their approach utilizes XLM-RoBERTa as an encoder with distinct regression heads for V and A. The core method involves a two-stage pretraining-finetuning process: first, a multilingual model (M') is pretrained, followed by fine-tuning (M'' l,d) on language- and domain-specific data. Experiments on a D20% train split of the SemEval-2026 Task 3 dataset showed that this two-stage fine-tuning improved performance by approximately 1.36 times in RMSEva compared to direct fine-tuning. The authors also analyzed system limitations and made their code publicly available.

Key takeaway

For NLP Engineers developing multilingual sentiment analysis systems, adopting a two-stage fine-tuning strategy can substantially boost performance. Your models, particularly those based on XLM-RoBERTa, could see an approximate 1.36 times improvement in RMSEva for dimensional aspect-based sentiment tasks like valence-arousal scoring. Consider implementing a pretraining phase followed by domain- and language-specific fine-tuning to enhance accuracy and robustness.

Key insights

A two-stage pretraining-finetuning approach significantly improves multilingual valence-arousal scoring using XLM-RoBERTa.

Principles

• Two-stage fine-tuning enhances DimABSA.
• Multilingual models benefit from domain adaptation.
• Separate heads for V/A improve prediction.

Method

The method involves pretraining a multilingual XLM-RoBERTa model, then fine-tuning it on domain- and language-specific data with separate regression heads for valence and arousal prediction.

In practice

• Implement two-stage fine-tuning for sentiment tasks.
• Use XLM-RoBERTa for multilingual text analysis.
• Consider separate regression heads for multi-dimensional output.

Topics

• Dimensional Sentiment Analysis
• Valence-Arousal Scoring
• Multilingual NLP
• XLM-RoBERTa
• Fine-tuning Strategies
• SemEval-2026 Task 3

Best for: Research Scientist, AI Scientist, NLP Engineer, Machine Learning Engineer

Related on AIssential

• Beyond Sentiment Classification: A Generative Framework for Emotion Intensity Evaluation in Text — Generative LLMs fine-tuned on continuous emotion intensity scores outperform classification and generalize to unseen affective dimensions.
• Aspect-Based Sentiment Analysis in the Accommodation Domain Using the BERTimbau Model — BERTimbau model fine-tuning strategies for Portuguese ABSA show trade-offs between recall and precision.
• mdok-style at SemEval-2026 Task 9: Finetuning LLMs for Multilingual Polarization Detection — Finetuning mid-sized LLMs with QLoRA and augmented data improves multilingual online polarization detection.
• Predicting relations between SOAP note sections: The value of incorporating a clinical information model — Fine-tuning a RoBERTa-base model on manually annotated clinical data improves NER for SOAP note sections.
• Use of Machine Learning techniques and Large Language Models for automatic evaluation of Celpe-Bras exam texts — BERT adaptations achieved the best automatic evaluation for Celpe-Bras texts, but at a higher computational cost.
• Multi-Stage Training for Abusive Comment Detection in Indic Languages — Multi-stage training with language preprocessing and model ensembles improves abusive comment detection while minimizing false positives.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.