PhysX-Omni: Unified Simulation-Ready Physical 3D Generation for Rigid, Deformable, and Articulated Objects

2026-05-22 · Source: cs.CV updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems, Gaming & Interactive Media · Depth: Expert, extended

Summary

PhysX-Omni is a unified framework for generating simulation-ready physical 3D assets, encompassing rigid, deformable, and articulated objects. Developed by S-Lab, Nanyang Technological University and ACE Robotics, it addresses limitations of prior methods by introducing a novel geometry representation tailored for Vision-Language Models, directly encoding high-resolution 3D structures without compression. The framework also includes PhysXVerse, the first general simulation-ready 3D dataset with over 8.7K assets across 2.9K indoor and outdoor categories, and PhysX-Bench, a new benchmark evaluating six key attributes: geometry, absolute scale, material, affordance, kinematics, and description. Extensive experiments show PhysX-Omni achieves superior performance, with a PSNR of 21.52, Chamfer Distance of 2.95, and F-score of 91.28 on PhysXVerse, and significantly reduced absolute scale errors (2.79). It also supports applications in robotic policy learning and scene generation.

Key takeaway

For robotics engineers and embodied AI developers building simulation environments, PhysX-Omni provides a powerful capability to generate diverse, physically accurate 3D assets. You can now directly integrate rigid, deformable, and articulated objects into your simulators, significantly reducing manual asset creation costs. This framework enables more realistic training scenarios and accelerates the development of robust robotic policies. Consider utilizing PhysX-Omni to enhance your simulation-based research and development.

Key insights

PhysX-Omni unifies simulation-ready 3D asset generation for diverse object types using a novel VLM-tailored geometry representation and comprehensive evaluation.

Principles

• Direct 3D structure modeling improves generation performance.
• Unified frameworks enhance asset diversity and applicability.
• Comprehensive benchmarks validate physical realism and generalization.

Method

PhysX-Omni uses a VLM-based coarse-to-fine global-to-local paradigm. It employs a template-based Run-Length Encoding (RLE) to directly model high-resolution 3D geometry, avoiding segmentation and enabling high-fidelity mesh generation with existing voxel decoders.

In practice

• Generate diverse rigid, deformable, and articulated objects.
• Create simulation-ready scenes from input images.
• Develop robotic policies with physically plausible generated assets.

Topics

• Simulation-Ready 3D Generation
• Vision-Language Models
• Physical 3D Assets
• Robotic Policy Learning
• PhysXVerse Dataset
• PhysX-Bench Benchmark

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, Robotics Engineer

Related on AIssential

• Mixing generative AI with physics to create personal items that work in the real world — PhysiOpt combines generative AI with physics simulations to create functional, real-world 3D-printable designs.
• Welcome NVIDIA Cosmos 3: The First Open Omni-model for Physical AI Reasoning and Action — NVIDIA Cosmos 3 unifies physical AI capabilities into a single omni-model for comprehensive world understanding and action generation.
• Phantom: Physics-Infused Video Generation via Joint Modeling of Visual and Latent Physical Dynamics — Integrating latent physical property inference into video generation improves physical plausibility and visual realism.
• Ex-Omni: Enabling 3D Facial Animation Generation for Omni-modal Large Language Models — Ex-Omni integrates 3D facial animation into OLLMs by decoupling semantic reasoning from temporal generation using speech units and TQGF.
• Integrate Physical AI Capabilities into Existing Apps with NVIDIA Omniverse Libraries — NVIDIA Omniverse now offers modular libraries for integrating physical AI capabilities into existing applications.
• SAM3D-Phys: Towards Multi-Object Interactive Simulation in Real World — SAM3D-Phys recovers complete, simulatable 3D object geometry from partial real-world scene observations for physics-based interaction.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CV updates on arXiv.org.