New Open Audio Models 🤗 | Recap with Jeff

2026-04-03 · Source: HuggingFace · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering, Cloud Computing & IT Infrastructure · Depth: Intermediate, short

Summary

Mistral has released Voxtral 4B, a new text-to-speech (TTS) model, noted for its speed and expressiveness, which can be tested on Hugging Face. Concurrently, Cohear launched Transcribe, a 2-billion parameter speech recognition model capable of converting speech to text across multiple languages. Transcribe is fast, highly efficient, and can run directly in a browser via Transformers.js, making it suitable for large-scale video content processing. Its permissive Apache 2.0 license further enhances its utility. To facilitate large-scale deployment of Cohear Transcribe, a UV script has been developed, leveraging Hugging Face's new features: Storage Buckets for AI-native data storage, HF Mount for streaming data access without full downloads, and HuggingFaceJobs for on-demand compute to run transcription tasks efficiently.

Key takeaway

For NLP Engineers evaluating new audio processing solutions, the release of Voxtral 4B and Cohear Transcribe presents significant opportunities. You should consider Cohear Transcribe for projects requiring high-speed, multi-language speech-to-text conversion, especially given its browser-based execution and Apache 2.0 license. For large-scale deployments, explore integrating Hugging Face's Storage Buckets, HF Mount, and HuggingFaceJobs to streamline your transcription workflows and manage massive datasets efficiently.

Key insights

New open audio models from Mistral and Cohear offer efficient text-to-speech and speech-to-text capabilities.

Principles

• Smaller models enable broader deployment.
• Permissive licenses foster widespread adoption.

Method

Deploy large-scale audio transcription by using Hugging Face Storage Buckets for data, HF Mount for streaming access, and HuggingFaceJobs for on-demand compute.

In practice

• Try Voxtral 4B for fast, expressive TTS.
• Use Cohear Transcribe for efficient, multi-language ASR.
• Utilize HF Mount for large dataset access.

Topics

• Open Audio Models
• Voxtral 4B TTS
• Cohear Transcribe ASR
• Hugging Face Storage Buckets
• HF Mount

Best for: NLP Engineer, AI Engineer, Machine Learning Engineer, MLOps Engineer

Related on AIssential

• Voxtral transcribes at the speed of sound — Mistral's Voxtral Transcribe 2 offers both open-weights and API-based speech-to-text models with real-time and diarization features.
• Now in Foundry: Cohere Transcribe, Nanbeige 4.1-3B, and Octen Embedding — Specialized, smaller open-source models can outperform larger general-purpose models across various AI tasks.
• Scenema Audio: Zero-shot expressive voice cloning and speech generation [N] — Scenema Audio offers zero-shot expressive voice cloning by decoupling emotional performance from voice identity using a diffusion model.
• How Mistral’s Voxtral TTS is Redefining Multilingual Voice Cloning with a Hybrid Autoregressive and Flow-Matching Architecture — Voxtral TTS uses a hybrid autoregressive and flow-matching architecture for high-fidelity multilingual voice cloning.
• Introducing VibeVoice ASR: Longform, Structured Speech Recognition At Scale — VibeVoice ASR unifies long-form speech transcription, diarization, and timestamping into a single, context-preserving model.
• Software for Automatic Speech Recognition via Whisper models applied to Oral History interviews in the Portuguese language — Ethos AT provides private, accessible automatic speech transcription for Portuguese using Whisper models.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by HuggingFace.