An Investigation Into Various Approaches For Bengali Long-Form Speech Transcription and Bengali Speaker Diarization

2026-03-03 · Source: Takara TLDR - Daily AI Papers · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Speech Recognition · Depth: Advanced, quick

Summary

This paper presents a multistage approach for Bengali long-form speech transcription and speaker diarization, addressing the challenges of this low-resource language for the "DL Sprint 4.0" Kaggle competitions. For transcription, the authors fine-tuned Whisper Medium on Bengali data, achieving a Word Error Rate (WER) of 0.38 on the private leaderboard through techniques like chunking and background noise cleaning. Speaker diarization utilized `pyannote/speaker-diarization-community-1` with a custom segmentation model and a two-pass method, resulting in a Diarization Error Rate (DER) of 0.27 on the private leaderboard and 0.19 on the public leaderboard. These results demonstrate that targeted tuning and strategic data utilization can significantly enhance AI inclusivity for South Asian languages, with all relevant code publicly available.

Key takeaway

For low-resource Bengali long-form speech, a multistage approach combining fine-tuned Whisper Medium and a two-pass pyannote diarization model achieved a 0.38 WER for transcription and 0.27 DER for speaker diarization. This demonstrates that strategic fine-tuning and data utilization can significantly enhance AI inclusivity for South Asian languages, providing practical solutions for complex audio processing.

Topics

• Bengali Speech Recognition
• Speaker Diarization
• Low-Resource Languages
• Whisper Model
• pyannote

Code references

• Short-Potatoes/Bengali-long-form-transcription-and-diarization

Best for: Machine Learning Engineer, AI Scientist, Research Scientist, AI Researcher, AI Engineer, NLP Engineer

Related on AIssential

• BaltiVoice: A Speech Corpus and Fine-tuned Whisper ASR System for the Balti Language — Fine-tuning pre-trained ASR models with modest, newly created corpora can establish functional speech recognition for low-resource languages.
• Mistral drops new speech-to-text AI models — Mistral's new speech-to-text models prioritize speed, privacy, and affordability for diverse enterprise applications.
• Multilingual Multi-Speaker Unit Vocoders: A Systematic Analysis of Discrete Speech Representations — Unit vocoder performance in multilingual settings hinges on cluster size for intelligibility and explicit speaker conditioning for identity preservation.
• Beyond Cascaded Pipelines: Building a Native Spoken Language Model Prototype — Native Spoken Language Models (SLMs) overcome cascaded pipeline limitations by directly processing audio, preserving nuance, reducing errors, and lowering latency.
• HybridCodec: Fast Dual-Stream, Semantically Enhanced Neural Audio Codec — HybridCodec combines dual-stream and distillation for fast, semantically specialized neural audio coding without SSL inference overhead.
• Voice of India: A Large-Scale Benchmark for Real-World Speech Recognition in India — Real-world Indic ASR requires benchmarks with unscripted speech, diverse languages, and flexible spelling.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Takara TLDR - Daily AI Papers.