Phonetic forced alignment for low-resource language varieties: Model training and evaluation on Chengdu Mandarin

2026-07-25 · Source: cs.CL updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics, Speech Technology & Phonetics · Depth: Expert, long

Summary

Researchers developed and evaluated specialized phonetic forced aligners for Chengdu Mandarin, a low-resource language variety with over 20 million speakers. They trained a text-dependent GMM-HMM model, Chengdu-MFA, using a 17-hour speech corpus and a custom grapheme-to-phoneme (G2P) dictionary. Subsequently, a pretrained audio encoder was fine-tuned as Chengdu-FC for text-independent alignment, leveraging pseudo-labels generated by Chengdu-MFA. Evaluation on an expert-annotated test set demonstrated that both Chengdu-MFA and Chengdu-FC significantly surpassed Standard Mandarin baselines. Chengdu-MFA reduced average phone boundary differences by 31.8%, while Chengdu-FC achieved a 61.2% reduction. This work provides a practical bootstrapping pipeline for creating accurate aligners for under-resourced language varieties without requiring labor-intensive manual annotation.

Key takeaway

For research scientists developing phonetic tools for under-resourced language varieties, this work offers a validated pipeline to overcome data scarcity. You should implement the G2P dictionary, GMM-HMM, and pseudo-labeling sequence to create accurate aligners. If utterance-level transcripts are available, prioritize the GMM-HMM model for high accuracy. For transcript-free segmentation, fine-tune a pretrained encoder using the generated pseudo-labels. This approach significantly reduces reliance on costly manual annotations.

Key insights

A bootstrapping pipeline effectively develops accurate phonetic aligners for low-resource language varieties using limited data.

Principles

• Variety-specific training improves phonetic alignment accuracy.
• GMM-HMMs are viable for low-resource text-dependent alignment.
• Pseudo-labels enable text-independent aligner training.

Method

Compile a custom G2P dictionary, train a GMM-HMM aligner, generate pseudo-labels, then fine-tune a pretrained speech encoder for text-independent alignment.

In practice

• Train custom G2P dictionaries for regional varieties.
• Apply GMM-HMM for text-dependent alignment.
• Use FC models for text-independent segmentation.

Topics

• Phonetic Forced Alignment
• Low-Resource Languages
• Chengdu Mandarin
• GMM-HMM Models
• Speech Encoders
• G2P Dictionary
• Pseudo-labeling

Best for: NLP Engineer, AI Scientist, Research Scientist

Related on AIssential

• Phonetic forced alignment for low-resource language varieties: Model training and evaluation on Chengdu Mandarin — A bootstrapping pipeline enables accurate phonetic forced alignment for low-resource language varieties using pseudo-labeling and fine-tuning.
• Scaling Human and G2P Supervision for Robust Phonetic Transcription — G2P supervision for phonetic transcription has diminishing returns beyond 20-30 hours of human annotation.
• Pretrained self-supervised speech models can recognize unseen consonants — Pretrained self-supervised speech models effectively generalize to and accurately recognize typologically rare phonemes, such as click consonants.
• Using embeddings to predict spoken word duration and pitch in Mandarin monosyllabic words — Contextualized embeddings effectively predict both duration and pitch contours for Mandarin monosyllabic words.
• An Investigation Into Various Approaches For Bengali Long-Form Speech Transcription and Bengali Speaker Diarization — This paper presents a multistage approach for Bengali long-form speech transcription and speaker diarization, addressing the challenges of this low-resource language for the "DL Sprint 4.0" Kaggle competitions.
• ReNikud: Audio-Supervised Hebrew Grapheme-to-Phoneme Conversion — ReNikud uses audio-supervised pseudo-labeling and a pseudo-vocalization architecture to improve Hebrew G2P conversion.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CL updates on arXiv.org.