Personalization in the Era of LLMs - Shivam Verma, Spotify

2026-05-19 · Source: AI Engineer · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Advanced, extended

Summary

Spotify is evolving its personalization system by integrating Large Language Models (LLMs) to move beyond traditional multi-step recommender pipelines. The company, which serves 750 million users with a catalog of over 100 million tracks and 400,000 audiobooks, is building a unified, steerable generative recommendation model. This new approach focuses on foundational user modeling, creating user embeddings that represent taste across historical interactions, and catalog understanding, which involves teaching LLMs about Spotify's content. Key to this is the use of "semantic IDs" to tokenize content vectors into 4-6 tokens, allowing LLMs to auto-regressively generate recommendations. The system also projects user representations into the LLM's token space as "soft tokens" to personalize outputs, moving towards a sequential modeling framework that enables features like the AI DJ and prompted playlists.

Key takeaway

For AI Architects and NLP Engineers building next-generation recommendation systems, Spotify's shift to LLM-backed generative models offers a blueprint. You should consider adopting semantic IDs for catalog understanding and projecting user embeddings as "soft tokens" into LLM prompts to achieve highly personalized and steerable recommendations, moving beyond traditional siloed models to a unified, sequential framework.

Key insights

Spotify is integrating LLMs into its recommendation system using user embeddings, semantic IDs, and soft tokens for enhanced personalization.

Principles

• Personalization requires deep user and content understanding.
• LLMs can be adapted for domain-specific recommendation tasks.
• User steerability enhances recommendation system utility.

Method

Spotify's method involves generating user embeddings, converting content vectors into semantic IDs for LLM training, and projecting user representations as soft tokens into the LLM's input for personalized generative recommendations.

In practice

• Implement semantic IDs to represent content for LLM integration.
• Use vector projection to personalize LLM outputs with user context.
• Explore transformer-based sequential models for recommendation.

Topics

• LLM Personalization
• Generative Recommender Systems
• User Embeddings
• Semantic IDs
• Transformer Models

Best for: AI Architect, NLP Engineer, AI Scientist, Machine Learning Engineer, AI Engineer, Director of AI/ML

Related on AIssential

• Rethinking Personalization for the Agent Era, Semantic Recall for Vector Search, and More! — LLMs are increasingly central to advancing information retrieval and recommender system capabilities.
• From Feelings to Metrics: Understanding and Formalizing How Users Vibe-Test LLMs — Formalized "vibe-testing" bridges the gap between LLM benchmark scores and real-world user utility.
• I Don’t Use Spotify. So I Built My Own AI DJ. — Building a personalized AI music curator requires overcoming API limitations and enriching diverse, often obscure, music metadata.
• 2025 Year in Review — Integrating LLMs with RecSys can yield powerful, steerable recommendation systems capable of cross-platform suggestions.
• A Foundation Model for Generative Recommendation, A Unified Definition of Hallucination, and More! — Recent research advances generative recommendation, LLM integration, and efficiency in information retrieval systems.
• LLMs Need Encoders for Semantic IDs Too — Semantic IDs in LLMs benefit from dedicated prefix-conditioned encoders like PrefixMem, similar to how multimodal LLMs use vision encoders.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by AI Engineer.