I Built an AI That Wrote Me a Country Breakup Song

2026-06-02 · Source: Siraj Raval · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Generative AI for Media · Depth: Intermediate, medium

Summary

An AI-generated country breakup song was created in 8 minutes using Fish Audio's S2 Pro, a platform enabling end-to-end text-to-song generation with advanced emotional control. The process involved recording 10-210 seconds of audio for voice cloning, then programmatically embedding natural language emotion tags like "whisper like a man who just lost his dog" into AI-generated lyrics. Fish Audio's S2 Pro leverages an open-source 4.4 billion parameter model, trained on 10 billion hours of audio data across 50 languages. This model employs a two-step architecture: a 4 billion parameter model for text and emotion input, feeding into a 400 million parameter model for raw audio waveform output. It utilizes Group Relative Policy Optimization (GRPO) for reinforcement learning, outperforming competitors like ElevenLabs and Inworld in benchmarks. The final song was produced by stitching the AI voice with a backing track in CapCut and uploaded to Spotify via RouteNote.

Key takeaway

For creative technologists or musicians exploring AI-driven content creation, Fish Audio's S2 Pro offers a powerful, rapid solution for producing emotionally rich, custom-voiced songs. You can clone your voice and embed nuanced emotions using natural language tags, significantly reducing production time to minutes. Consider integrating this platform into your workflow to quickly prototype or finalize tracks, leveraging its advanced capabilities to differentiate your AI-generated audio. This technology changes how you approach personalized audio content.

Key insights

Fish Audio's S2 Pro enables rapid, emotionally nuanced AI song generation via voice cloning and natural language emotion tagging.

Principles

• Emotion control is key for advanced AI audio.
• Two-model architecture enhances generation accuracy.
• Post-training innovation drives AI audio progress.

Method

Record 10-210 seconds for voice cloning. Input lyrics with natural language emotion tags. Generate audio, then stitch with backing track. Upload to distribution platforms.

In practice

• Use Fish Audio for custom voice generation.
• Experiment with natural language emotion tags.
• Combine AI audio with external tools like CapCut.

Topics

• AI Music Generation
• Voice Cloning
• Emotion AI
• Fish Audio S2 Pro
• Reinforcement Learning
• Audio Deep Learning

Best for: AI Engineer, AI Student, Creative Technologist

Related on AIssential

• Fish Audio Review — Fish Audio provides robust AI voice generation with extensive models and API, suitable for high-volume content, though ElevenLabs leads in emotional nuance.
• 😺 Watch: This AI just made our podcast theme song — Google's Flow Music, powered by Lyria 3, enables iterative AI-driven music and video creation.
• Amazon’s new Alexa+ powered feature can generate podcast episodes — Alexa+ now generates on-demand, customizable podcast episodes using AI voices and real-time information.
• SUNO 5.5 INSANITY and other AI news... — Suno's new voice cloning and custom model features advance AI music generation, blurring lines between human and AI-created sound.
• Spotify takes on Google’s NotebookLM with its new app — Spotify's Studio app generates personalized AI podcasts from user data and web content, expanding its audio strategy.
• Google DeepMind Releases Lyria 3: An Advanced Music Generation AI Model that Turns Photos and Text into Custom Tracks with Included Lyrics and Vocals — Lyria 3 is a multimodal AI generating high-fidelity music with vocals and lyrics from text and photos.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Siraj Raval.