OccSim: Multi-kilometer Simulation with Long-horizon Occupancy World Models

2026-04-01 · Source: cs.CV updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems · Depth: Expert, extended

Summary

OccSim is presented as the first occupancy world model-driven 3D simulator designed to overcome the limitations of data-driven autonomous driving simulations that rely on pre-recorded driving logs or HD maps. It can generate over 3,000 continuous frames, constructing large-scale 3D occupancy maps spanning over 4 kilometers from a single initial frame and future ego-actions, representing an >80x improvement in stable generation length. OccSim utilizes two modules: W-DiT for ultra-long-horizon static environment generation by incorporating rigid transformations, and a Layout Generator for populating dynamic foregrounds with reactive agents based on synthesized road topology. Data collected from OccSim can pre-train 4D semantic occupancy forecasting models, achieving up to 67% zero-shot performance on unseen data, outperforming previous asset-based simulators by 11%. Scaling the dataset 5x increases zero-shot performance to 74%, expanding the improvement to 22.1%.

Key takeaway

For research scientists developing autonomous driving systems, OccSim offers a novel approach to generate high-fidelity, large-scale simulation data without reliance on traditional maps or logs. You should consider integrating OccSim's generated datasets for pre-training 4D semantic occupancy forecasting models, as it significantly improves zero-shot performance compared to asset-based simulators, enabling more robust model development.

Key insights

OccSim is an occupancy world model-driven 3D simulator enabling multi-kilometer, log-independent autonomous driving simulations.

Principles

• Decouple static and dynamic voxel generation.
• Explicitly use 3D geometric priors for stability.
• Augment training data to prevent overfitting.

Method

OccSim uses a W-DiT module for static scene generation with rigid transformations and a Layout Generator for dynamic agent population, followed by A* routing and 2D-IDM for agent control.

In practice

• Pre-train 4D semantic occupancy models with OccSim data.
• Generate diverse, multi-kilometer road networks.
• Simulate reactive agents without historical logs.

Topics

• Occupancy World Models
• Autonomous Driving Simulation
• W-DiT Architecture
• Long-horizon Generation
• 4D Semantic Occupancy Forecasting

Code references

• Orbis36/OccSim

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, Robotics Engineer

Related on AIssential

• VISA: VLM-Guided Instance Semantic Auditing for 3D Occupancy World Models — VISA uses offline VLMs as reliability-aware auditors to improve 3D occupancy world models' semantic accuracy during training.
• The Sequence Knowledge #825: Inside World Labs Marble — World Labs' Marble model, founded by Dr. Fei-Fei Li, represents a significant advancement in AI, shifting focus from traditional "temporal prediction" of pixels to spatial intelligence. Designated as…
• OmniLiDAR: A Unified Diffusion Framework for Multi-Domain 3D LiDAR Generation — OmniLiDAR unifies 3D LiDAR generation across diverse sensing domains using a text-conditioned diffusion framework.
• SeeThrough3D: Occlusion Aware 3D Control in Text-to-Image Generation — Explicitly modeling occlusions in 3D scene representations enhances text-to-image generation realism and control.
• 🦪OccAny: Universal 3D Occupancy🦪 👉OccAny by Valeo is a novel unified framework for... — OccAny is a unified framework for generalized, unconstrained urban 3D occupancy prediction.
• OmniTraffic: A Controllable Generation Pipeline and Benchmark for Spatio-Temporal Traffic Reasoning — OmniTraffic is a benchmark and pipeline for spatio-temporal traffic reasoning, revealing significant MLLM gaps in complex traffic understanding.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CV updates on arXiv.org.