IIITK_SpeechScape@DravidianLangTech 2026: Dialect based speech recognition and classification using Speech Foundation Models and Deep Learning Techniques

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, short

Summary

IIITK_SpeechScape's work at DravidianLangTech 2026 addresses dialectal variation challenges in Automatic Speech Recognition (ASR) and dialect classification for low-resource, morphologically rich languages like Tamil. Tamil, spoken across India, Sri Lanka, and the diaspora, exhibits significant phonetic, lexical, and prosodic differences across its dialects. The research evaluates state-of-the-art models, including Whisper, CLDNN, wav2vec, and wavLM for dialect classification, and Whisper and a zero-shot Conformer for ASR, within a unified framework. Whisper demonstrated the best performance, achieving a macro F1-score of 0.46 for dialect classification and a word error rate of 0.57 for ASR. These results underscore the strong generalization capabilities of transformer-based foundation models across diverse dialects and languages. The project's code is publicly available on GitHub.

Key takeaway

For NLP Engineers and AI Scientists developing speech technologies for low-resource, dialectally rich languages like Tamil, you should prioritize evaluating transformer-based foundation models such as Whisper. Its demonstrated performance, with a 0.46 macro F1-score for dialect classification and 0.57 WER for ASR, suggests a robust baseline. Utilize the publicly available code to accelerate your research and development efforts in similar challenging linguistic contexts.

Key insights

Transformer-based foundation models like Whisper show strong generalization for dialectal ASR and classification in low-resource languages.

Principles

• Dialectal variation significantly challenges ASR.
• Transformer models generalize across dialects.
• Low-resource languages benefit from foundation models.

Method

The work evaluates state-of-the-art models (Whisper, CLDNN, wav2vec, wavLM, Conformer) for dialect classification and ASR within a unified framework, specifically for Tamil.

In practice

• Consider Whisper for Tamil ASR tasks.
• Explore the public GitHub code.
• Evaluate foundation models for dialect classification.

Topics

• Automatic Speech Recognition
• Dialect Classification
• Tamil Language
• Speech Foundation Models
• Whisper Model
• Low-Resource Languages
• Deep Learning Techniques

Best for: AI Engineer, Research Scientist, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• BaltiVoice: A Speech Corpus and Fine-tuned Whisper ASR System for the Balti Language — Fine-tuning pre-trained ASR models with modest, newly created corpora can establish functional speech recognition for low-resource languages.
• Software for Automatic Speech Recognition via Whisper models applied to Oral History interviews in the Portuguese language — Ethos AT provides private, accessible automatic speech transcription for Portuguese using Whisper models.
• Speech Emotion Recognition Leveraging OpenAI's Whisper Representations and Attentive Pooling Methods — Whisper's pre-trained ASR representations, combined with attention-based pooling, significantly enhance Speech Emotion Recognition.
• Speech-Driven End-to-End Language Discrimination towards Chinese Dialects — Speech-driven features, integrating MFCC and word embeddings through a CNN, significantly improve Chinese dialect discrimination.
• A Comparative Study of Pretrained Transformer Models for Quranic ASR: Speech Representations, Label Formats, and Dataset Composition — Fine-tuning pretrained Transformers with specific speech representations and label formats significantly improves Quranic ASR accuracy and efficiency.
• Beyond Cascaded Pipelines: Building a Native Spoken Language Model Prototype — Native Spoken Language Models (SLMs) overcome cascaded pipeline limitations by directly processing audio, preserving nuance, reducing errors, and lowering latency.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.