CHMOD_777@DravidianLangTech 2026: Tamil-Adapted Whisper and MMS for Dialect Speech Recognition and Classification

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Natural Language Processing · Depth: Expert, medium

Summary

Team CHMOD_777 developed a system for the DravidianLangTech@ACL 2026 shared task, focusing on Tamil dialect speech recognition and classification. This system addresses two subtasks: classifying Tamil speech into four regional dialects (Northern, Southern, Western, Central) and transcribing dialectal Tamil speech to text. For dialect classification, the team fine-tuned MMS-1b-all using Focal Loss and weighted sampling, achieving an 83.04 Macro F1 score on the development set, placing 5th among 11 teams. For speech recognition, a Tamil-specific Whisper model with 763M parameters was fine-tuned, resulting in a 53.72 WER on the development set and a 49.75 WER on the official test set, securing 1st place out of 13 teams. A significant finding is that this domain-specific Tamil Whisper model, despite having half the parameters (763M), outperformed the larger general-purpose Whisper-large-v3 (1.5B) by 8 WER points.

Key takeaway

For NLP Engineers developing speech recognition or classification systems for low-resource or dialectal languages, you should prioritize domain-specific model adaptation. Your efforts in fine-tuning smaller, specialized models like Tamil Whisper (763M parameters) can yield superior performance, beating larger general-purpose models such as Whisper-large-v3 (1.5B) by substantial margins (e.g., 8 WER points). Consider applying techniques like Focal Loss and weighted sampling for robust dialect classification.

Key insights

Domain-specific pre-training significantly enhances speech model performance over larger general-purpose alternatives.

Principles

• Domain-specific models can surpass larger general models.
• Fine-tuning with Focal Loss improves dialect classification.

Method

Fine-tuning MMS-1b-all with Focal Loss and weighted sampling for classification; fine-tuning a Tamil-specific Whisper model for speech recognition.

In practice

• Adapt Whisper models for specific language domains.
• Use Focal Loss for imbalanced dialect datasets.

Topics

• Tamil Speech Recognition
• Dialect Classification
• Whisper Model
• MMS-1b-all
• Domain Adaptation
• Focal Loss

Best for: AI Engineer, Research Scientist, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• BaltiVoice: A Speech Corpus and Fine-tuned Whisper ASR System for the Balti Language — Fine-tuning pre-trained ASR models with modest, newly created corpora can establish functional speech recognition for low-resource languages.
• Speech-Driven End-to-End Language Discrimination towards Chinese Dialects — Speech-driven features, integrating MFCC and word embeddings through a CNN, significantly improve Chinese dialect discrimination.
• Phonetic Error Analysis of Raw Waveform Acoustic Models — Raw waveform models with BLSTMs and transfer learning improve phone recognition, with errors reflecting inherent phonetic similarities.
• Beyond Cascaded Pipelines: Building a Native Spoken Language Model Prototype — Native Spoken Language Models (SLMs) overcome cascaded pipeline limitations by directly processing audio, preserving nuance, reducing errors, and lowering latency.
• Speech Emotion Recognition Leveraging OpenAI's Whisper Representations and Attentive Pooling Methods — Whisper's pre-trained ASR representations, combined with attention-based pooling, significantly enhance Speech Emotion Recognition.
• A Comparative Study of Pretrained Transformer Models for Quranic ASR: Speech Representations, Label Formats, and Dataset Composition — Fine-tuning pretrained Transformers with specific speech representations and label formats significantly improves Quranic ASR accuracy and efficiency.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.