From RNNs to Transformers: Building Sequential Recommenders (Part 1)

2026-04-18 · Source: MLWhiz: Recs|ML|GenAI · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Intermediate, quick

Summary

This article, "The Generative Recommender Revolution, Part 1," traces the evolution of sequential recommendation systems, highlighting the shift from traditional classification-based approaches to generative models. It begins by formalizing the sequential problem, where the order of user interactions (e.g., [i₁, i₂, i₃. .., iₙ]) is crucial for predicting the next item (iₙ₊₁), unlike collaborative filtering which treats interactions as an unordered bag. The post uses the Steam Games dataset, comprising 56,808 users, 6,382 games, and 2,235,453 interactions after 5-core filtering, as a benchmark. It then details GRU4Rec (2016), the first major work applying Recurrent Neural Networks (RNNs) to session-based recommendations, which significantly influenced the field for two years.

Key takeaway

For AI Architects and Machine Learning Engineers designing recommendation systems, understanding the transition from traditional collaborative filtering to sequential and generative models is critical. Your current pipeline, which might involve multi-stage retrieval and ranking, could be simplified and improved by exploring generative recommenders that directly predict the next item. Consider evaluating GRU4Rec as a foundational sequential model, especially for session-based contexts, to enhance prediction accuracy by accounting for interaction order.

Key insights

Sequential recommendation models predict the next item by considering the order of past user interactions.

Principles

• Order of interactions matters for next-item prediction.
• Generative models can subsume classification tasks.

Method

Sequential recommendation models predict iₙ₊₁ given [i₁, i₂, i₃. .., iₙ], applicable across e-commerce, streaming, music, and news domains.

In practice

• Use 5-core filtering for RecSys datasets.
• Apply GRU4Rec for session-based recommendations.

Topics

• Sequential Recommendation
• Generative Recommenders
• GRU4Rec
• SASRec
• BERT4Rec

Best for: Machine Learning Engineer, AI Engineer, AI Architect

Related on AIssential

• Implicit Reasoning for LLM-based Generative Recommendation, Why Stronger Encoders Make Weaker SPLADE Models, and More! — Recent research focuses on refining LLM-based recommendation and retrieval systems through diverse architectural and algorithmic improvements.
• [D] Advice on sequential recommendations architectures — Sequential recommendation challenges often stem from representation and objective mismatches, not just architecture.
• OneReason Technical Report — Effective generative recommendation reasoning requires both itemic token perception and user behavior cognition.
• Empirical Patterns in Real-World Agentic Search Sessions, How Many Dimensions Do You Really Need for Top-k Retrieval?, and More! — Recent advances in information retrieval focus on agentic search, robust recommendation, and efficient embedding techniques.
• TokenFormer: Unify the Multi-Field and Sequential Recommendation Worlds — TokenFormer unifies multi-field and sequential recommendation by mitigating "Sequential Collapse Propagation" through novel attention and non-linear interaction mechanisms.
• Agentic Recommender System with Hierarchical Belief-State Memory — Hierarchical memory and adaptive lifecycle management significantly enhance agentic recommender system performance.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by MLWhiz: Recs|ML|GenAI.