Track2Map: Online Deformable SLAM with Motion-Aware Pose Optimization in Robotic Surgery

2026-07-09 · Source: Artificial Intelligence · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems · Depth: Expert, quick

Summary

Track2Map is an online 3D Gaussian Splatting pipeline designed for robot-assisted minimally invasive surgery (RAMIS), addressing limitations of existing offline methods that rely on accurate camera trajectory priors. This system jointly optimizes camera trajectory and a deformable 3D scene representation directly from surgical video, enabling robust reconstructions even when camera priors are absent or noisy. It functions as a Simultaneous Localization and Mapping (SLAM) method. To stabilize optimization amidst tissue motion and ambiguous visual cues, Track2Map incorporates a track-anchored deformation initialization using dense 2D point tracks. It further utilizes track statistics to differentiate camera motion from scene deformation, detecting static camera periods and reducing drift during incremental mapping. Experiments on the StereoMIS dataset demonstrate improved reconstruction quality and camera trajectory accuracy compared to both competing SLAM and non-SLAM methods.

Key takeaway

For Robotics Engineers developing surgical navigation or 3D reconstruction systems, Track2Map offers a significant advancement by providing robust online deformable SLAM. You should consider integrating similar joint optimization and motion-aware techniques to overcome challenges posed by noisy or absent camera trajectory priors in dynamic surgical environments. This approach can enhance the accuracy and reliability of your systems, improving their applicability in real-world robot-assisted minimally invasive surgery.

Key insights

Track2Map enables robust online deformable 3D reconstruction in robotic surgery by jointly optimizing camera pose and scene deformation.

Principles

• Joint optimization improves robustness in dynamic surgical scenes.
• Track-anchored deformation initialization stabilizes optimization.
• Track statistics disentangle camera motion from scene deformation.

Method

Track2Map uses dense 2D point tracks for deformation initialization and leverages track statistics to detect static camera periods, reducing drift during incremental mapping for online 3D Gaussian Splatting.

In practice

• Implement online SLAM for deformable surgical reconstruction.
• Utilize 2D point tracks to initialize deformation models.
• Apply track statistics to separate camera and scene motion.

Topics

• Robotic Surgery
• Deformable SLAM
• 3D Gaussian Splatting
• Camera Trajectory Optimization
• Minimally Invasive Surgery
• Surgical Navigation

Best for: Computer Vision Engineer, AI Scientist, Robotics Engineer, Research Scientist

Related on AIssential

• Track2Map: Online Deformable SLAM with Motion-Aware Pose Optimization in Robotic Surgery — Track2Map enables robust online deformable 3D reconstruction and camera localization in RAMIS without relying on accurate trajectory priors.
• Track2Map: Online Deformable SLAM with Motion-Aware Pose Optimization in Robotic Surgery — Track2Map robustly performs online deformable SLAM in surgical video by disentangling camera motion from tissue deformation.
• SplatCtrl: Perception-Action Coupling via Gaussian Scene Representations and Reactive Robot Control — SplatCtrl unifies 3D Gaussian Splatting with control barrier functions for real-time, collision-free robot control in dynamic scenes.
• Video Friday: Your Robot Surgeon Will See You Now — Robotics advancements are pushing boundaries in surgical applications, generalist learning, and complex locomotion.
• Benchmarking Action Spaces in Reinforcement Learning for Vision-based Robotic Manipulation — Joint velocity action space significantly improves sim-to-real performance for vision-based robotic picking and pushing tasks.
• MatterDoor: Sampling Zero-shot Spatio-semantic Priors using Generative Models — Generative models can sample spatio-semantic priors for robotic planning in partially observed 3D environments.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Artificial Intelligence.