Stereo2Spatial: Convert Stereo Music Tracks to Spatialized Binaural Mixes [P]

2026-07-17 · Source: Machine Learning · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering · Depth: Expert, short

Summary

The Stereo2Spatial model, developed over approximately six months, converts standard stereo music tracks into spatialized binaural mixes. Initially, a flow-matching diffusion model was built in the latent space of an EAR-VAE, encoding stereo input into one latent and 7.1.4 output channels. This latent approach, inspired by ImmersiveFlow, included memory tokens for stable long-context generations but faced quality bottlenecks due to VAE distribution mismatch. Pivoting to a raw waveform model resolved quality issues but introduced training instability, with loss shooting up after 60K-80K steps. This was overcome by implementing "amplitude lifting" from the WavFlow paper, scaling audio tracks to an RMS of 0.33 and multiplying by 3, using a clip of 4.0 for a model space of -12 to 12. The waveform model was trained on 7,669 tracks for about 20 days on 2x A6000 GPUs, with optional mix-style conditioning. A Windows desktop application is also available for inference.

Key takeaway

For AI Engineers or audio developers exploring spatial audio generation, the Stereo2Spatial model offers a robust open-source solution for converting stereo to binaural mixes. You should consider integrating this Apache 2.0 licensed model, especially noting the effectiveness of amplitude lifting for training stability and the availability of a Windows inference app. Evaluate its direct binaural output and explore mix-style conditioning for your projects.

Key insights

Converting stereo to spatial audio via flow-matching diffusion is feasible, with raw waveform modeling and amplitude lifting crucial for quality and stability.

Principles

• Raw waveform modeling improves audio quality over latent space.
• Amplitude lifting stabilizes flow-matching diffusion training.
• Memory tokens enable long-context audio generation.

Method

The waveform model trains in two stages on 7,669 tracks, using amplitude lifting (RMS 0.33, clip 4.0) to stabilize flow-matching diffusion.

In practice

• Use the provided Windows app for local inference.
• Explore mix-style conditioning for output control.
• Contribute CPU optimizations to the GitHub repo.

Topics

• Spatial Audio
• Binaural Audio
• Flow-Matching Diffusion
• Waveform Modeling
• Amplitude Lifting
• Open-Source AI

Code references

• Eps-Acoustic-Revolution-Lab/EAR_VAE
• francislabountyjr/stereo2spatial
• francislabountyjr/stereo2spatial-app

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, AI Engineer

Related on AIssential

• Memory Efficient Audio Synthesis with Decoupled Temporal Depth Diffusion Transformers — Decoupling temporal and depth processing in Diffusion Transformers enables highly memory-efficient, high-fidelity on-device audio synthesis.
• NeuralMUSIC: A Hybrid Neural-Subspace Framework for Robot Sound Source Localization — NeuralMUSIC combines neural networks with classical MUSIC for robust, generalizable robot sound source localization.
• LiveBand: Live Accompaniment Generation in the Audio Domain — LiveBand generates real-time, high-fidelity music accompaniments using a causal transformer in an audio autoencoder's latent space, improving quality and alignment.
• WanSong v1.0 Technical Report — Pure diffusion models can generate high-fidelity, long-form, multi-stem music efficiently with customization support.
• DirectAudioEdit: Inversion-Free Text-Guided Audio Editing via Diffusion Prediction Contrast — DirectAudioEdit enables efficient, inversion-free text-guided audio editing, outperforming inversion-based methods in speed and quality.
• Scenema Audio: Zero-shot expressive voice cloning and speech generation [N] — Scenema Audio offers zero-shot expressive voice cloning by decoupling emotional performance from voice identity using a diffusion model.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Machine Learning.