cclin at SemEval-2026 Task 2 : SLM-Enhanced Lightweight Multi-BERT Ensemble for Longitudinal Affect Assessment

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Natural Language Processing · Depth: Expert, quick

Summary

Team cclin developed an efficient hybrid system for SemEval-2026 Task 2, Subtask 1: Longitudinal Affect Assessment, aiming to predict Valence and Arousal from text over time. The system integrates quantized 7B-scale language models as meta-feature extractors with an ensemble of DeBERTa, RoBERTa, and DistilBERT encoders. Designed to operate on a single consumer-grade RTX 5060 Ti (16GB) GPU, it achieved competitive results, ranking 3rd out of 31 teams. The model was trained as an ordinal classification problem, using expected-value decoding for continuous score prediction. It attained an overall V&A average of 0.587, with composite correlations of 0.647 for Valence and 0.527 for Arousal. This approach demonstrates a strong performance–efficiency trade-off through lightweight SLM-derived priors and multi-encoder fusion.

Key takeaway

For Machine Learning Engineers developing affect assessment systems with resource constraints, consider adopting a hybrid architecture. You can achieve competitive Valence and Arousal prediction by combining quantized 7B-scale language models for meta-features with an ensemble of smaller BERT-family encoders. This approach allows deployment on consumer-grade GPUs like the RTX 5060 Ti (16GB), offering a strong performance-efficiency balance. Implement ordinal classification with expected-value decoding to handle continuous output.

Key insights

Lightweight SLM-enhanced multi-BERT ensembles offer efficient, competitive longitudinal affect assessment.

Principles

• Quantized SLMs can serve as efficient meta-feature extractors.
• Ensemble methods improve affect prediction, especially for Arousal.
• Ordinal classification with expected-value decoding bridges discrete/continuous.

Method

The system uses quantized 7B-scale LMs for meta-features, combined with a DeBERTa, RoBERTa, and DistilBERT ensemble. It trains as ordinal classification, decoding probabilities to continuous scores.

In practice

• Deploy affect assessment models on 16GB GPUs like RTX 5060 Ti.
• Combine small LMs with BERT-family encoders for efficiency.
• Use expected-value decoding for continuous output from classification.

Topics

• Longitudinal Affect Assessment
• Valence Arousal Prediction
• SLM-Enhanced Ensembles
• Quantized Language Models
• DeBERTa RoBERTa DistilBERT
• GPU-Efficient Inference

Best for: AI Engineer, Research Scientist, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• How to Fine-Tune an SLM for Emotion Recognition — Fine-tuning SLMs for fine-grained emotion recognition on imbalanced datasets requires a multi-technique approach to data balancing and loss weighting.
• From Affect Prediction to Affect Forecasting: Evidence for Distinct Information Sources in Longitudinal Text — Current affect prediction relies on text semantics, while future affective change forecasting benefits from numeric trajectory dynamics.
• TeamHerald@CHIPSAL 2026: Hate Speech Detection and Sentiment Analysis of Nepali Memes using Transformer-based Architectures and Ensemble Learning — Ensemble strategies for meme analysis perform differently for binary vs. multi-class tasks, requiring task-specific aggregation.
• Deep Temporal Modeling and Ensemble Fusion for Multimodal Emotion Recognition from Physiological Signals — Combining deep temporal models with sensor and ensemble fusion significantly boosts physiological emotion recognition accuracy.
• Working Notes on Late Interaction Dynamics: Analyzing Targeted Behaviors of Late Interaction Models — Late Interaction models exhibit length bias, especially in causal multi-vector architectures, while MaxSim effectively uses top token similarity.
• Facial-Expression-Aware Prompting for Empathetic LLM Tutoring — Integrating structured facial expression data via prompt engineering significantly boosts LLM tutor empathy without retraining.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.