UAReviews: A Multi-Task Ukrainian Dataset for Emotion and Intent Classification

2026-06-28 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Expert, short

Summary

UAReviews is a new multi-task Ukrainian-language dataset designed for emotion and intent classification, comprising 11,580 annotated texts. This dataset uniquely combines citizen reviews of government digital services from Ukraine's Ministry of Digital Transformation with Ukrainian-language Telegram posts sourced from the COSMUS corpus. Each text within UAReviews is dually annotated with an emotion label, adhering to the seven classes of the Ekman taxonomy, and an intent label, categorized into five distinct classes. This makes UAReviews the first publicly available Ukrainian resource specifically for joint emotion and intent analysis. The annotation process involved students, with a 20% gold standard subset rigorously validated by three independent annotators, achieving a high Krippendorff's alpha of 0.93. Baseline performance has been established using single-task and multi-task fine-tuned XLM-RoBERTa models, alongside an analysis of emotion-to-intent correlation. Both the UAReviews dataset and its baseline models are publicly available.

Key takeaway

For NLP Engineers developing Ukrainian language models, UAReviews provides a critical resource for advancing emotion and intent classification. You should integrate this publicly available dataset to train and benchmark multi-task models, using its dual annotations for Ekman emotions and specific intents. This enables more nuanced understanding of user feedback and communication patterns in Ukrainian, improving model performance and application accuracy. Consider analyzing the emotion-to-intent correlation to refine your model's predictive capabilities.

Key insights

The UAReviews dataset offers the first public Ukrainian resource for joint emotion and intent classification, validated with high inter-annotator agreement.

Principles

• Multi-task annotation enhances linguistic resource utility.
• High inter-annotator agreement ensures dataset reliability.
• Combining diverse text sources enriches dataset scope.

Method

Texts from government reviews and Telegram posts are dually annotated for 7 Ekman emotions and 5 intents. A 20% gold standard subset was validated by three annotators, achieving Krippendorff's alpha = 0.93.

In practice

• Fine-tune XLM-RoBERTa for Ukrainian NLP tasks.
• Analyze emotion-intent correlation in user feedback.
• Develop multi-task models for complex text analysis.

Topics

• Ukrainian NLP
• Emotion Classification
• Intent Classification
• Multi-task Learning
• Dataset Annotation
• XLM-RoBERTa

Best for: Research Scientist, AI Scientist, NLP Engineer, AI Student

Related on AIssential

• Acoustic Cue Alignment in Audio Language Models for Speech Emotion Recognition — Audio Language Models integrate explicit acoustic concept tokens with raw audio, improving Speech Emotion Recognition without over-relying on symbolic cues.
• Accelerating researchers and developers building multilingual AI with a new open dataset — The GitHub Multilingual Repositories Dataset provides metadata to discover non-English content in over 40 million public repositories.
• From Fallback to Frontline: When Can LLMs be Superior Annotators of Human Perspectives? — LLMs can statistically outperform human annotators in estimating aggregate subgroup opinions under common conditions.
• Ukrainian Visual Word Sense Disambiguation Benchmark — Multimodal LLMs significantly underperform in Ukrainian Visual-WSD compared to English, revealing a critical language resource gap.
• A Context-Aware Dataset for Stance Detection in Bioethical Controversies on Reddit — BioStance offers a large, context-aware dataset for stance detection in complex bioethical social media discourse.
• 30% of Google's Emotions Dataset is Mislabeled — Poor data labeling methodology, especially lacking context and cultural fluency, severely compromises dataset quality and model performance.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.