CHMOD_777@DravidianLangTech 2026: LLM Augmented Transformer Fine-tuning for Tamil Political Sentiment Analysis

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Expert, quick

Summary

Team CHMOD_777 developed a system for the DravidianLangTech@ACL 2026 shared task, focusing on political multiclass sentiment analysis of Tamil Twitter comments. The challenge involved classifying tweets into seven sentiment categories despite a severe 8:1 class imbalance. Their approach utilized LLM-based data augmentation with Gemini 2.5 Flash, expanding the training data from 4,352 to 15,316 samples, a 3.5x increase. The best system, MuRIL fine-tuned on this augmented data using Focal Loss (gamma=3.0) and weighted sampling, achieved 35.79% Macro F1 on the development set, representing a 67% relative improvement over a non-augmented baseline. On the official test set, the system scored 34.25% Macro F1, ranking 12th among 22 teams. Key findings indicate that language-specific pre-training like MuRIL (236M) outperforms larger general models such as IndicBERT-v3 (1B), smaller models benefit significantly from augmentation, and the "Substantiated" category (F1=10.7%) proved hardest due to its factual reasoning requirement.

Key takeaway

For NLP Engineers tackling political sentiment analysis in low-resource languages like Tamil, you should prioritize LLM-based data augmentation using models such as Gemini 2.5 Flash to address severe class imbalance. Your model selection should favor language-specific pre-trained Transformers, even smaller ones, as they can outperform larger general models when combined with augmentation. Implement Focal Loss and weighted sampling to further enhance performance on imbalanced datasets.

Key insights

LLM-based data augmentation significantly improves sentiment analysis performance for imbalanced, low-resource language tasks.

Principles

• Language-specific models can outperform larger general models.
• Smaller models benefit disproportionately from data augmentation.
• Factual reasoning categories pose significant sentiment analysis challenges.

Method

LLM-based data augmentation using Gemini 2.5 Flash, followed by MuRIL Transformer fine-tuning with Focal Loss (gamma=3.0) and weighted sampling.

In practice

• Augment training data with LLMs like Gemini 2.5 Flash.
• Employ Focal Loss and weighted sampling for class imbalance.
• Consider language-specific models for low-resource NLP.

Topics

• Tamil Sentiment Analysis
• LLM Data Augmentation
• Transformer Fine-tuning
• Class Imbalance
• MuRIL Model
• Political Text Analysis

Best for: AI Scientist, NLP Engineer, Research Scientist

Related on AIssential

• mdok-style at SemEval-2026 Task 9: Finetuning LLMs for Multilingual Polarization Detection — Finetuning mid-sized LLMs with QLoRA and augmented data improves multilingual online polarization detection.
• Is Sentiment Analysis in Qualitative Data Analysis Software Accurate? — Sentiment analysis accuracy in QDA tools varies significantly, with LLMs offering superior performance at higher costs.
• Graph-Augmented LLMs for Swiss MP Ideology Prediction — Graph-augmented LLMs can model complex political ideologies from text and relational data.
• mdok-style at SemEval-2026 Task 10: Finetuning LLMs for Conspiracy Detection — Data augmentation and self-training can effectively finetune LLMs for specialized text classification tasks with limited data.
• Beyond Acoustic Emotion Recognition: Multimodal Pathos Analysis in Political Speech Using LLM-Based and Acoustic Emotion Models — LLM-based multimodal analysis captures political emotion better than acoustic models, despite acoustic utility for Arousal.
• Comparing LLM and Fine-Tuned Model Performance on NVDRS Circumstance Extraction with Varying Prompt Complexity — LLMs, guided by a context-aware "Complexity Score" for prompt selection, significantly outperform fine-tuned models on inferentially complex, low-prevalence data extraction.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.