Introducing SAM Audio: The First Unified Multimodal Model for Audio Separation | AI at Meta

2025-12-16 · Source: AI at Meta · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Intermediate, quick

Summary

Meta has introduced SAM Audio, a new multimodal model designed for audio separation across music, speech, and general soundscapes. This model allows users to isolate specific audio elements using either text prompts or visual prompts. It supports advanced features such as span prompts for enhanced precision and the ability to combine multiple prompts to refine the separation workflow. SAM Audio is presented as a versatile tool with broad applications for professionals and enthusiasts in fields like music production, audio engineering, and video creation, offering a unified approach to complex audio isolation tasks.

Key takeaway

For audio engineers and content creators working with complex soundscapes, SAM Audio offers a powerful new capability to isolate specific audio elements. You should explore its text and visual prompting features to streamline your workflow for tasks like vocal extraction or sound effect isolation, potentially reducing manual editing time significantly.

Key insights

SAM Audio unifies multimodal audio separation using text or visual prompts for diverse applications.

Principles

• Multimodal input enhances audio separation.
• Prompt engineering refines sound isolation.

Method

SAM Audio separates audio by interpreting text prompts, visual cues, or a combination thereof, including span prompts for precise targeting, to isolate specific sounds from complex audio.

In practice

• Isolate vocals from music tracks.
• Extract specific sound effects from video.

Topics

• SAM Audio
• Audio Separation
• Multimodal AI
• Text Prompts
• Visual Prompts

Best for: AI Engineer, Machine Learning Engineer, Creative Technologist, AI Product Manager

Related on AIssential

• SAM Audio - AI at Meta — SAM Audio offers state-of-the-art sound separation using multimodal prompts across diverse audio types.
• Meta’s SAM Audio Explained (And Why It Matters) — Meta's SAM Audio model enables precise sound isolation from media using text prompts, offering significant utility for audio manipulation.
• Acoustic Prompting via Stage-wise Modulation for Few-Shot Learning in Audio Language Models — Integrating trainable audio-side prompts with text-side prompt tuning significantly enhances few-shot adaptation in Audio-Language Models.
• New open-source voice model listens nonstop and decides every 0.4 seconds whether to speak or stay silent — The "Audio Interaction" model unifies continuous audio processing, multi-tasking, and proactive response in a single system.
• AI Audio Updates: Spotify, Particle, ElevenLabs — AI audio innovations are reshaping media consumption, offering personalized experiences and new monetization models.
• Dialogue Boost: How Amazon is using AI to enhance TV and movie dialogue — AI-driven audio separation enhances dialogue clarity by suppressing background sounds, improving accessibility and viewing experience.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by AI at Meta.