Solaris: Building a Multiplayer Video World Model in Minecraft

2026-02-25 · Source: Computer Vision and Pattern Recognition · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems, Data Science & Analytics · Depth: Advanced, quick

Summary

Solaris is a novel multiplayer video world model designed to simulate consistent multi-view observations, addressing the limitations of existing single-agent action-conditioned video generation models. To facilitate its development, a new multiplayer data system was created for robust, continuous, and automated data collection within video games like Minecraft. This system supports coordinated multi-agent interaction and synchronized capture of videos and actions, a significant advancement over prior single-player platforms. Utilizing this system, 12.64 million multiplayer frames were collected, and an evaluation framework was proposed to assess multiplayer movement, memory, grounding, building, and view consistency. Solaris is trained via a staged pipeline that transitions from single-player to multiplayer modeling, incorporating bidirectional, causal, and Self Forcing training, with Checkpointed Self Forcing introduced for memory-efficient, longer-horizon teaching.

Key takeaway

For research scientists developing video world models, Solaris demonstrates a critical shift from single-agent to multi-agent simulation. You should consider adopting a multiplayer data collection system and a staged training approach, including Checkpointed Self Forcing, to effectively model complex, interactive environments and achieve consistent multi-view observations in your next-generation models.

Key insights

Solaris is a multiplayer video world model that simulates consistent multi-view observations using a novel data collection and training system.

Principles

• Multi-agent interactions require multi-view observation models.
• Staged training improves complex model development.

Method

Solaris uses a staged training pipeline, progressively moving from single-player to multiplayer modeling, combining bidirectional, causal, and Checkpointed Self Forcing techniques for long-horizon teaching.

In practice

• Use a multiplayer data system for multi-agent environments.
• Employ Checkpointed Self Forcing for memory-efficient training.

Topics

• Multiplayer Video World Models
• Minecraft Simulation
• Multi-agent Data Collection
• Self Forcing Training
• Multi-agent Evaluation

Best for: Computer Vision Engineer, Research Scientist, AI Researcher, AI Scientist, Machine Learning Engineer

Related on AIssential

• Micro-World: First AMD Open-Source World Models for Interactive Video Generation — Micro-World is an open-source, action-controlled world model for interactive video generation, trained on Minecraft data.
• Multi-agent Framework for Time-Sensitive Complementary Collaboration in Minecraft — Large language models struggle with real-time, dynamic multi-agent collaboration due to latency and coordination complexity.
• AI is taking over video games — Eve Online's complex, player-driven environment offers unique data for training AI in long-term planning.
• ProPlay: Procedural World Models for Self-Evolving LLM Agents — ProPlay uses a procedural world model and graph to enable LLM agents to self-evolve by rehearsing and refining future task paths.
• World Models Are Here—But It’s Still the GPT-2 Phase — World models offer continuous, interactive simulations, learning world evolution from vast video data, currently in an exploratory GPT-2 phase.
• PhysEditWorld: A Large-Scale Dataset Toward Physics-Editable World Models — PhysEditWorld is a multimodal dataset enabling physics-editable world models through controlled physical parameter variation.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computer Vision and Pattern Recognition.