From a Multilingual Streaming ASR Backbone to Kenyan-Language Systems: Data-Centric Adaptation of Nemotron 3.5 for Kikuyu, Dholuo, and Kalenjin

2026-07-21 · Source: Computation and Language · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering · Depth: Expert, quick

Summary

NVIDIA Nemotron 3.5 ASR Streaming 0.6B was adapted for Kikuyu, Dholuo, and Kalenjin, addressing challenges like orthographic inconsistency and annotation artifacts in African languages. This end-to-end engineering study began with a Kenyan Swahili-adapted checkpoint, employing full-parameter fine-tuning while preserving its cache-aware FastConformer RNN-T, prompt conditioning, and streaming decoder. The adaptation process involved comprehensive steps including corpus auditing, Unicode normalization, duration filtering, and true-streaming evaluation. On internal evaluation sets, selected Kikuyu and Dholuo models achieved 42.97% and 33.98% Word Error Rates, respectively, with Dholuo also recording 9.59% CER. Kalenjin remains in progress, showing 68.74% WER on a 2,411-row diagnostic subset. The study also reported negative findings regarding non-speech labels and cloud job failures, emphasizing an auditable account of adapting a multilingual streaming model to language-specific systems under streaming constraints.

Key takeaway

For Machine Learning Engineers adapting large ASR models to low-resource languages, this study highlights the critical need for a data-centric approach. You should meticulously audit corpora, perform Unicode normalization, and implement true-streaming evaluation to ensure robust system performance. Your adaptation strategy must retain streaming constraints and account for orthographic inconsistencies and annotation artifacts to achieve reliable deployment-ready systems, even if it means foregoing public benchmark comparisons.

Key insights

Data-centric adaptation of a multilingual streaming ASR model to low-resource languages requires meticulous engineering and auditing.

Principles

• Orthographic inconsistency and annotation artifacts constrain ASR for African languages.
• Retain streaming constraints during adaptation for deployment relevance.
• Internal evaluation sets and normalization differences preclude public benchmark claims.

Method

Full-parameter fine-tuning of a cache-aware FastConformer RNN-T, prompt conditioning, and streaming decoder, preceded by corpus auditing, Unicode normalization, and duration filtering.

In practice

• Audit corpora for consistency and artifacts.
• Perform Unicode normalization and split checks.
• Use validation-based checkpoint selection.

Topics

• Automatic Speech Recognition
• Nemotron 3.5
• Multilingual ASR
• Data-Centric Adaptation
• Low-Resource Languages
• African Languages

Best for: AI Engineer, Research Scientist, Machine Learning Engineer, NLP Engineer, AI Scientist

Related on AIssential

• A data-centric approach to performance improvement in under-resourced ASR: The case of Dënë Sųłıné — Data quality, especially transcription consistency, is crucial for low-resource ASR performance.
• MoLGE: Mixture of Language Group Experts for Efficient Scaling of Massively Multilingual Speech Recognition — MoLGE uses language grouping and LoRA to efficiently scale multilingual ASR, overcoming capacity dilution.
• Pushing the Limits of On-Device Streaming ASR: A Compact, High-Accuracy English Model for Low-Latency Inference — On-device ASR can achieve high accuracy and low latency on CPU through systematic benchmarking and quantization.
• Scaling ASR for Hutsul Dialect: Multi-Speaker Data Collection, Enhanced Transcription and Cross-Speaker Evaluation — Scaling ASR for low-resource dialects requires multi-speaker data and a RAG-enhanced transcription pipeline.
• Data Scale, Not Latency, Shapes Cross-Lingual Encoder Transfer in Streaming ASR — Multilingual initialization benefits streaming ASR primarily in low-data regimes, not due to latency constraints.
• Diarization-Guided Qwen-ASR Adaptation for Multilingual Two-Speaker Conversational Speech — A diarization-guided, multi-stage ASR adaptation system significantly improves multilingual two-speaker conversational speech recognition.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computation and Language.