Transformer-based segmentation of prosodic boundaries in Brazilian Portuguese

2026-07-08 · Source: Computation and Language · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering, Natural Language Processing · Depth: Expert, quick

Summary

SAMPA, a novel Whisper-based segmenter, automatically identifies and marks terminal prosodic boundaries in Brazilian Portuguese (BP) speech. This system addresses a gap where BP segmentation largely relies on rule-based or traditional machine learning methods, unlike English which benefits from recent deep learning approaches. Researchers fine-tuned Whisper large-v3 using manually segmented recordings from the NURC-SP dataset. Evaluation included various training and test-time filtering configurations, alongside out-of-distribution testing on the MuPe-Diversidades dataset. SAMPA demonstrated competitive boundary-detection performance, achieving an F1 score of 0.731 on its held-out test split and 0.796 on MuPe-Diversidades. Analyses confirmed the model utilizes morphosyntactic, semantic, and prosodic cues for accurate boundary detection.

Key takeaway

For NLP Engineers developing speech processing applications for Brazilian Portuguese, SAMPA demonstrates a robust deep learning approach for prosodic boundary segmentation. You should consider fine-tuning large pre-trained models like Whisper for similar low-resource language tasks, as this method achieves competitive F1 scores (0.731-0.796). This can significantly improve the naturalness of speech synthesis and the accuracy of automatic speech recognition systems for BP.

Key insights

SAMPA, a Whisper-based segmenter, effectively identifies prosodic boundaries in Brazilian Portuguese speech using deep learning.

Principles

• Fine-tuning large pre-trained models adapts them to specific linguistic tasks.
• Out-of-distribution testing validates model robustness across datasets.
• Prosodic boundary detection relies on morphosyntactic, semantic, and prosodic cues.

Method

Fine-tune Whisper large-v3 on manually segmented speech data, then evaluate with various filtering configurations and out-of-distribution datasets.

In practice

• Transcribe BP speech with explicit prosodic boundary markers.
• Improve speech synthesis naturalness for Brazilian Portuguese.
• Enhance automatic speech recognition for BP.

Topics

• Prosodic Segmentation
• Brazilian Portuguese
• Whisper Model
• Speech Transcription
• Deep Learning
• Language Models

Best for: AI Scientist, NLP Engineer

Related on AIssential

• Transformer-based segmentation of prosodic boundaries in Brazilian Portuguese — Prosodic segmentation for Brazilian Portuguese can be effectively achieved by fine-tuning Whisper large-v3 to embed boundary markers in transcriptions.
• IIITK_SpeechScape@DravidianLangTech 2026: Dialect based speech recognition and classification using Speech Foundation Models and Deep Learning Techniques — Transformer-based foundation models like Whisper show strong generalization for dialectal ASR and classification in low-resource languages.
• Robustness and Diversity Evaluation on ProsSegue-ML: a Free Prosodic Segmentation Tool for Brazilian Portuguese — ProsSegue-ML's robustness and bias are evaluated for Brazilian Portuguese prosodic segmentation.
• Speech-Guided Multimodal Learning for Vocal Tract Segmentation in Real-Time MRI — Multimodal supervision during training can enhance single-modality inference for challenging medical image segmentation tasks.
• Bridging the Gap: Converting Read Text to Conversational Dialogue — PACC converts read speech to natural conversational speech using deep neural networks and HiFi-GAN for prosodic adjustment.
• Using embeddings to predict spoken word duration and pitch in Mandarin monosyllabic words — Contextualized embeddings effectively predict both duration and pitch contours for Mandarin monosyllabic words.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computation and Language.