AI4PC-Howard University at SemEval-2026 Task 2: Fine-Tuning DistilBERT, DeBERTa and ModernBERT for Valence–Arousal Prediction and Change Estimation

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics, Natural Language Processing · Depth: Expert, short

Summary

AI4PC-Howard University developed lightweight, reproducible models for longitudinal valence–arousal (VA) prediction within the SemEval-2026 Task 2 essay corpus. Their approach utilized only official data, enforcing user-disjoint splits to prevent data leakage. The team evaluated three distinct settings: essay-level VA state estimation, short-horizon VA change forecasting, and long-horizon disposition change prediction. For essay-level regression (Subtask 1), they employed DistilBERT. Short-horizon change forecasting (Subtask 2A) involved ModernBERT-based history modeling with a GRU and a blended previous-delta baseline. For long-horizon disposition change prediction (Subtask 2B), pooled DeBERTa history embeddings with a compact MLP were used. On the official evaluation, their best approaches achieved rcomp =0.665/0.468 for valence/arousal in Subtask 1, r = 0.597/0.413 for Subtask 2A, and r =0.046/0.348 for Subtask 2B.

Key takeaway

For NLP Engineers developing longitudinal sentiment analysis systems, consider these lightweight BERT-based architectures for valence-arousal prediction. Your data splitting strategy must enforce user-disjoint splits to ensure robust model generalization and prevent leakage. Evaluate distinct models for essay-level state, short-term change, and long-term disposition, as different architectures like DistilBERT, ModernBERT+GRU, and DeBERTa+MLP show varied performance across these specific tasks.

Key insights

Lightweight models fine-tuned on BERT variants predict longitudinal valence-arousal in essays across three distinct tasks.

Principles

• Enforce user-disjoint data splits to prevent leakage.
• Utilize official data exclusively for reproducibility.

Method

The approach involves fine-tuning DistilBERT for essay-level VA regression, combining ModernBERT with a GRU for short-horizon change, and using pooled DeBERTa embeddings with an MLP for long-horizon disposition change.

In practice

• Fine-tune DistilBERT for essay-level VA regression.
• Combine ModernBERT with GRU for short-horizon change.

Topics

• Valence-Arousal Prediction
• Longitudinal Sentiment Analysis
• SemEval-2026 Task 2
• BERT Fine-Tuning
• DistilBERT
• DeBERTa

Best for: Research Scientist, AI Scientist, NLP Engineer

Related on AIssential

• Sentiment Analysis with BERT: The Heart of the Transformer, The Soul of the Text — BERT's bidirectional context understanding via Transformers significantly advanced sentiment analysis and NLP tasks.
• From Affect Prediction to Affect Forecasting: Evidence for Distinct Information Sources in Longitudinal Text — Current affect prediction relies on text semantics, while future affective change forecasting benefits from numeric trajectory dynamics.
• Can Machines Tell Whether Gamers Love or Hate a Game? — Fine-tuning Transformer models like RoBERTa significantly improves sentiment analysis accuracy on imbalanced, complex human language datasets.
• Aspect-Based Sentiment Analysis in the Accommodation Domain Using the BERTimbau Model — BERTimbau model fine-tuning strategies for Portuguese ABSA show trade-offs between recall and precision.
• WARBERT: A Hierarchical BERT-based Model for Web API Recommendation — WARBERT is a hierarchical BERT model that combines filtering and matching for efficient, accurate Web API recommendations.
• Working Notes on Late Interaction Dynamics: Analyzing Targeted Behaviors of Late Interaction Models — Late Interaction models exhibit length bias, especially in causal multi-vector architectures, while MaxSim effectively uses top token similarity.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.