CHMOD_777@DravidianLangTech 2026: Context-Aware Fine-tuned MuRIL for Abusive Tamil Text Detection on Social Media

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Expert, medium

Summary

Team CHMOD_777 developed a system for the DravidianLangTech@ACL 2026 shared task, focusing on detecting abusive Tamil text targeting women on social media. Their approach involved fine-tuning three transformer backbones: MuRIL, XLM-RoBERTa, and IndicBERT-v3, utilizing Focal Loss and weighted sampling. The team systematically evaluated the impact of context length, hyperparameter tuning, and language-specific pre-training. Their top-performing system, MuRIL with a 256-token context, achieved an 82.76% Macro F1 score on the development set. It also secured 80.61% on the official test set, ranking 6th among 24 participating teams. Key findings indicate that increasing context from 128 to 256 tokens improved F1 and accelerated convergence by 2.4x. The language-specific MuRIL model (236M parameters) outperformed the larger IndicBERT (270M parameters). Furthermore, default hyperparameters proved optimal, as tuning attempts consistently degraded performance.

Key takeaway

For NLP Engineers building abusive text detection systems for Dravidian languages, you should prioritize context length and language-specific models. Extending your model's context to 256 tokens can significantly improve F1 scores and accelerate training convergence by 2.4x. Focus on fine-tuning models like MuRIL (236M parameters) rather than larger, more general models (e.g., IndicBERT, 270M parameters). Furthermore, rigorously test default hyperparameters, as aggressive tuning might degrade your system's performance.

Key insights

Context length and language-specific pre-training are critical for effective abusive text detection, often outperforming larger models or hyperparameter tuning.

Principles

• Longer context (256 tokens) improves F1 and speeds convergence.
• Language-specific models (e.g., MuRIL) can outperform larger general models.
• Default hyperparameters may be optimal; tuning can degrade performance.

Method

Fine-tune transformer backbones (MuRIL, XLM-RoBERTa, IndicBERT-v3) using Focal Loss and weighted sampling, systematically evaluating context length and pre-training.

In practice

• Use 256-token context for faster, better F1 scores.
• Prioritize language-specific pre-trained models over larger general ones.
• Test default hyperparameters before extensive tuning.

Topics

• Abusive Text Detection
• Tamil Language Processing
• Transformer Fine-tuning
• MuRIL Model
• Context Length
• Hyperparameter Optimization

Best for: AI Engineer, Research Scientist, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• Multi-Stage Training for Abusive Comment Detection in Indic Languages — A multi-stage pipeline combining language preprocessing and model ensembles effectively detects abusive Indic language comments while minimizing false positives.
• How Small Can You Go? LoRA Fine-Tuning 270M-8B Models for Merchant Information Extraction in Financial Transactions — Smaller LoRA-fine-tuned models can achieve near 8B performance for structured extraction, significantly reducing deployment costs.
• A Comparative Study of Pretrained Transformer Models for Quranic ASR: Speech Representations, Label Formats, and Dataset Composition — Fine-tuning pretrained Transformers with specific speech representations and label formats significantly improves Quranic ASR accuracy and efficiency.
• MAAM: Anchor-Preserving Compression and Contextual Calibration for Chinese Discriminatory Language Detection — MAAM uses anchor preservation and contextual calibration for robust, compact Chinese discriminatory language detection.
• mdok-style at SemEval-2026 Task 9: Finetuning LLMs for Multilingual Polarization Detection — Finetuning mid-sized LLMs with QLoRA and augmented data improves multilingual online polarization detection.
• IndicContextEval: A Benchmark for Evaluating Context Utilisation in Audio Large Language Models Across 8 Indic Languages — AudioLLMs vary significantly in how they utilize contextual prompts, with native-script entities and natural language being most effective.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.