URDF Synthesis from RGB-D Sequences via Differentiable Joint Inference and Energy-Consistent Verification

2026-06-17 · Source: Artificial Intelligence · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems · Depth: Expert, quick

Summary

KinemaForge is a novel pipeline designed to reconstruct simulation-ready digital twins of articulated objects from short RGB-D sequences. It addresses key limitations in current methods, specifically the decoupling of part-level geometry from kinematic parameter estimation and violations of dynamic invariants leading to simulation drift. KinemaForge jointly infers part-level shape, joint topology, and joint parameters, validating results with an energy-consistent verifier built on differentiable rigid-body dynamics. The pipeline integrates a kinematic constraint graph, a differentiable screw-axis solver utilizing Featherstone's articulated-body algorithm, and an energy residual loss function. Evaluated across five PartNet-Mobility categories, KinemaForge reduces average joint-axis error from 4.52 degrees to 2.83 degrees (-37.4%) over PARIS and from 5.30 degrees to 2.83 degrees (-46.6%) over Ditto. It also lowers long-horizon simulation drift by 64% over 50-second rollouts and improves closed-loop manipulation success by 14.6 percentage points.

Key takeaway

For Robotics Engineers developing digital twins or physics simulations, KinemaForge offers a significant advancement in creating accurate URDFs from RGB-D data. You should consider integrating its principles of joint inference and energy-consistent verification to mitigate simulation drift and improve manipulation task success. This approach can yield more robust and reliable models for complex robotic systems, reducing iterative adjustments and enhancing real-world performance.

Key insights

KinemaForge jointly infers articulated object kinematics and geometry, ensuring energy consistency for accurate simulation.

Principles

• Joint inference improves digital twin accuracy.
• Energy-consistent verification prevents simulation drift.
• Differentiable dynamics enables joint parameter optimization.

Method

KinemaForge uses a kinematic constraint graph, a differentiable screw-axis solver with Featherstone's algorithm, and an energy residual loss to infer and validate URDFs from RGB-D.

In practice

• Generate simulation-ready digital twins.
• Improve robotic manipulation success rates.
• Reduce long-horizon physics simulation drift.

Topics

• URDF Synthesis
• RGB-D Reconstruction
• Differentiable Dynamics
• Kinematic Inference
• Digital Twins
• Robotic Manipulation
• Featherstone's Algorithm

Best for: Research Scientist, Computer Vision Engineer, AI Scientist, Robotics Engineer, Machine Learning Engineer

Related on AIssential

• GEM-4D: Geometry-Enhanced Video World Models for Robot Manipulation — Distilling 4D geometry foundation model features into video world models ensures correspondence-consistent generation for robot manipulation.
• 🤖Physically-Plausible Human🤖 👉PhysMoDPO is a novel direct preference optimization... — PhysMoDPO uses direct preference optimization to generate physically plausible humanoid motions.
• MimicIK: Real-Time Generative Inverse Kinematics from Teleoperation with FK Consistency — MimicIK uses conditional flow matching and FK consistency for robust, real-time generative inverse kinematics from teleoperation.
• RoboLayout: Differentiable 3D Scene Generation for Embodied Agents — RoboLayout generates agent-navigable 3D scenes by integrating explicit reachability constraints and local refinement into differentiable optimization.
• Real2Sim: A Physics-driven and Editable Gaussian Splatting Framework for Autonomous Driving Scenes — Real2Sim combines 4D Gaussian Splatting with a differentiable MPM solver for physics-aware, editable autonomous driving scene generation.
• Japan Science and Technology Agency Develops NVIDIA-Powered Moonshot Robot for Elderly Care — Robots can predict 3D interaction forces from RGB images using simulation-trained AI.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Artificial Intelligence.