PICT at SemEval-2026 Task 3: A Transformer-Based System for Dimensional Aspect-Aware Sentiment Regression with Weighted Layer Pooling

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics, Natural Language Processing · Depth: Expert, quick

Summary

Team PICT's submission for SemEval-2026 Task 3, named DimASR, addresses continuous valence and arousal prediction. Their system focuses on reducing variance and preventing negative transfer across domains. It employs strictly domain-isolated pipelines for the Laptop and Restaurant datasets, utilizing a RoBERTa-Large backbone. The architecture incorporates weighted layer pooling to extract a rich feature hierarchy, an aspect-aware attention module driven by the [CLS] token for local context isolation, and a deep residual regression head for mapping to continuous space. Regularized with R-Drop and SWA, the system achieved 3rd place in the Restaurant domain with an RMSE of 1.195 and 9th place in the Laptop domain with an RMSE of 1.326.

Key takeaway

For NLP Engineers developing sentiment regression models, consider implementing domain-isolated pipelines to prevent negative transfer, especially when dealing with distinct datasets like product reviews. Your systems could benefit from a RoBERTa-Large backbone combined with weighted layer pooling and [CLS]-driven aspect-aware attention for improved performance. Applying regularization techniques like R-Drop and SWA can further enhance model stability and accuracy, potentially improving your competitive standing in tasks like DimASR.

Key insights

Domain-isolated Transformer systems with weighted layer pooling and aspect-aware attention excel at dimensional sentiment regression, reducing variance.

Principles

• Focus on variance reduction.
• Avoid cross-domain negative transfer.
• Isolate local context with attention.

Method

Build domain-isolated pipelines using RoBERTa-Large. Extract features via weighted layer pooling, apply [CLS]-driven aspect-aware attention, and map to continuous space with a deep residual regression head, regularized by R-Drop and SWA.

In practice

• Use RoBERTa-Large for sentiment tasks.
• Implement weighted layer pooling.
• Apply R-Drop and SWA for regularization.

Topics

• Dimensional Sentiment Regression
• Transformer Models
• RoBERTa-Large
• Weighted Layer Pooling
• Aspect-Aware Attention
• SemEval-2026

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• DiffuSent: Towards a Unified Diffusion Framework for Aspect-Based Sentiment Analysis — DiffuSent unifies ABSA subtasks via non-auto-regressive boundary denoising, improving accuracy and speed for multi-word sentiment extraction.
• Automatic Reflection Level Classification in Hungarian Student Essays — Automated reflection classification in Hungarian essays is feasible using both classical ML and fine-tuned transformers.
• Turkish Sentiment Analysis (Keras LSTM + BERT) — A hybrid NLP pipeline combines custom LSTMs and pre-trained BERT for robust Turkish sentiment analysis with real-time data acquisition.
• AspectRAG: An Architecture of Retrieval and Generation for Aspect-Based Sentiment Analysis — AspectRAG leverages aspect-guided retrieval and generation for unsupervised, high-performance sentiment analysis in Portuguese.
• Zero-Shot to Full-Resource: Cross-lingual Transfer Strategies for Aspect-Based Sentiment Analysis — Cross-lingual transfer strategies for ABSA vary in effectiveness based on model architecture and resource availability.
• GHI: Graphormer over Conditioned Hypergraph Incidence for Aspect-Based Sentiment Analysis — GHI unifies diverse linguistic and semantic evidence into token–hyperedge incidence relations for robust ABSA.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.