Slot-MPC: Goal-Conditioned Model Predictive Control with Object-Centric Representations

2026-05-14 · Source: Machine Learning · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems · Depth: Expert, quick

Summary

Slot-MPC is a novel object-centric world modeling framework designed for goal-conditioned Model Predictive Control (MPC) in robotic manipulation tasks. It utilizes vision encoders to learn slot-based representations, which individually encode objects within a scene. These structured representations then facilitate the learning of an action-conditioned object-centric dynamics model. During inference, this learned dynamics model enables action planning via MPC, allowing agents to adapt to novel situations. The framework employs gradient-based MPC for action optimization, which offers greater computational efficiency compared to gradient-free, sampling-based MPC methods. Experimental results on simulated robotic manipulation tasks demonstrate that Slot-MPC enhances both task performance and planning efficiency when compared to non-object-centric world model baselines.

Key takeaway

For research scientists developing robotic manipulation systems, Slot-MPC offers a path to more generalizable and efficient action planning. You should consider integrating object-centric world models with gradient-based MPC to improve performance in novel situations, especially when working with limited state-action coverage in offline settings.

Key insights

Slot-MPC combines object-centric representations with gradient-based MPC for efficient, generalizable robotic action planning.

Principles

• Object-centric representations provide strong inductive bias.
• Differentiable world models enable gradient-based MPC.

Method

Slot-MPC learns slot-based object representations via vision encoders, then an action-conditioned dynamics model, enabling gradient-based MPC for planning and adaptation.

In practice

• Apply slot-based representations for scene understanding.
• Use gradient-based MPC for efficient action optimization.

Topics

• Slot-MPC
• Object-Centric Representations
• Model Predictive Control
• Gradient-Based MPC
• Robotic Manipulation

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, Robotics Engineer

Related on AIssential

• HOLO-MPPI: Multi-Scenario Motion Planning via Hierarchical Policy Optimization — HOLO-MPPI integrates high-level learned policies with low-level stochastic optimal control for robust multi-scenario motion planning.
• Gradient-based planning for world models at longer horizons — GRASP enables robust long-horizon planning for world models by parallelizing optimization and improving gradient signals.
• Direction-Conditioned Policies via Compositional Subgoal Scoring for Online Goal-Conditioned Reinforcement Learning — Optimal goal-conditioned actions depend on the goal's value gradient, not raw goal states.
• Mem-World: Memory-Augmented Action-Conditioned World Models for Persistent Robot Manipulation — Mem-World uses a 4D surfel-indexed memory for geometry-aware history retrieval to enable persistent robot manipulation.
• Decoupled Object-Centric Video Understanding for Generating Robotic Manipulation Commands — Decoupling action recognition from object identification improves robotic command generation from video.
• Making Foresight Actionable: Repurposing Representation Alignment in World Action Models — Aligning visual and action representations in World Action Models improves robot manipulation performance and robustness.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Machine Learning.