TeleAI at SemEval-2026 Task 3: Large Language Models for Dimensional Aspect-Based Sentiment Analysis

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Expert, quick

Summary

TeleAI's system for SemEval-2026 Task 3, Track A, Subtask 1 (DimASR) addresses the challenge of predicting continuous Valence-Arousal (VA) scores for specific text aspects. The framework frames this as an end-to-end regression problem, utilizing Qwen2.5-7B as its feature extraction backbone. Parameter-efficient fine-tuning is achieved via LoRA. To enhance generalization and mitigate domain shifts, the system employs multilingual and multi-domain mixed training. Further optimization and robustness techniques include R-Drop-style consistency regularization, embedding-level PGD adversarial training, Smooth L1 (Huber) loss, sigmoid-based output interval mapping, and post-hoc linear calibration. Comprehensive ablations confirm that combining joint training with robustness regularizations significantly reduces the official $RMSE_{VA}$ metric, demonstrating competitive performance across diverse language and domain settings for lightweight LLM adaptation in dimensional aspect-based sentiment analysis.

Key takeaway

For Machine Learning Engineers developing dimensional aspect-based sentiment analysis systems, you should consider lightweight LLM adaptation with robust training. Leveraging Qwen2.5-7B and LoRA, combined with multilingual/multi-domain mixed training and techniques like R-Drop or PGD adversarial training, can significantly improve continuous Valence-Arousal score prediction. This approach offers a competitive path to enhance model generalization and stability, reducing $RMSE_{VA}$ across diverse language and domain settings.

Key insights

Lightweight LLM adaptation via LoRA and robust training techniques effectively predicts continuous Valence-Arousal scores for text aspects.

Principles

• Multilingual/multi-domain training improves generalization.
• Robustness techniques stabilize continuous predictions.
• LoRA enables efficient LLM adaptation.

Method

Frame dimensional aspect-based sentiment analysis as end-to-end regression. Use Qwen2.5-7B with LoRA, multilingual/multi-domain training, R-Drop, PGD adversarial training, Smooth L1 loss, sigmoid mapping, and post-hoc calibration.

In practice

• Fine-tune Qwen2.5-7B with LoRA for efficiency.
• Employ mixed training to reduce domain shift.
• Implement R-Drop and PGD for robust regression.

Topics

• Dimensional Sentiment Analysis
• Large Language Models
• LoRA Fine-tuning
• Qwen2.5-7B
• Multilingual Training
• Adversarial Training

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• System Report for CCL25-Eval Task 5: New Dataset and LoRA-Fine-Tuned Qwen2.5 — Domain-specific LoRA fine-tuning on Qwen2.5-14B with CCPoetry-49K significantly enhances classical Chinese poetry understanding and translation.
• Bayesian-LoRA: Probabilistic Low-Rank Adaptation of Large Language Models — Bayesian-LoRA improves LLM calibration and uncertainty by integrating probabilistic low-rank adaptation with normalizing flows.
• A Lightweight LLM Framework for Disaster Humanitarian Information Classification — Parameter-efficient fine-tuning significantly outperforms prompting and RAG for disaster tweet classification, enabling efficient, accurate crisis intelligence.
• A NATO-accredited organization published a report concerning language-based disparities in LLM performance (Published April 25, 2026) — LLM performance varies significantly by language, but adaptation strategies can bridge the gap.
• LoRA: Understanding Low-Rank Adaptation — LoRA approximates full weight updates using low-rank matrices for efficient, latency-free fine-tuning.
• Auditing Training Data in Domain-adapted LLMs: LoRA-MINT — LoRA-MINT effectively audits training data membership in LoRA-adapted LLMs by analyzing perplexity.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.