Track2Map: Online Deformable SLAM with Motion-Aware Pose Optimization in Robotic Surgery

2026-07-11 · Source: cs.AI updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems · Depth: Expert, long

Summary

Track2Map is an online 3D Gaussian Splatting pipeline designed for dense, deformable 3D anatomy reconstruction in robot-assisted minimally invasive surgery (RAMIS). It jointly optimizes camera trajectory and deformable scene representation directly from surgical video, addressing the limitations of offline methods that rely on accurate, often noisy or absent, camera pose priors. The system functions as a Simultaneous Localization and Mapping (SLAM) method, employing a track-anchored deformation initialization using dense 2D point tracks. It further utilizes track statistics to differentiate camera motion from tissue deformation by identifying static camera periods, thereby reducing drift during incremental mapping. Evaluated on StereoMIS, Track2Map demonstrated improved reconstruction quality and camera trajectory accuracy compared to existing SLAM methods like Endo3R and EndoGSLAM, and non-SLAM approaches such as Online-endo-track [8], even under absent or noisy pose priors. The system uses CoTracker3 for 2D tracking and runs at 6 seconds per frame on an NVIDIA Tesla V100.

Key takeaway

For Robotics Engineers developing computer guidance techniques in RAMIS, you can achieve robust deformable 3D mapping without relying on precise robot kinematics. Track2Map demonstrates that leveraging stereo depth, consistent 2D correspondences, and motion-aware pose optimization significantly improves reconstruction quality and camera trajectory, even with noisy or absent pose priors. Consider integrating these principles into your online SLAM systems to enhance stability in dynamic surgical environments. Be mindful of the current 6 seconds per frame processing time, which may necessitate keyframe processing for real-time applications.

Key insights

Track2Map robustly performs online deformable SLAM in surgical video by disentangling camera motion from tissue deformation.

Principles

• Coherent global flow indicates camera motion.
• Intermittent endoscope motion allows gated pose updates.
• Stereo depth grounds metric 3D reconstruction.

Method

Track2Map computes a motion score from 2D flow direction dispersion to gate pose updates. It lifts 2D correspondences to 3D anchors for deformation initialization, then jointly refines scene and pose with various losses.

In practice

• Utilize CoTracker3 for robust 2D point tracking.
• Implement motion gating to prevent pose drift.
• Integrate stereo depth for consistent metric scale.

Topics

• Robotic Surgery
• Deformable SLAM
• 3D Gaussian Splatting
• Motion-Aware Optimization
• Surgical Video Analysis
• CoTracker3

Best for: Computer Vision Engineer, AI Scientist, Robotics Engineer, Research Scientist

Related on AIssential

• Track2Map: Online Deformable SLAM with Motion-Aware Pose Optimization in Robotic Surgery — Track2Map enables robust online deformable 3D reconstruction and camera localization in RAMIS without relying on accurate trajectory priors.
• DL-SLAM: Enabling High-Fidelity Gaussian Splatting SLAM in Dynamic Environments based on Dual-Level Probability — DL-SLAM uses a dual-level probabilistic framework combining semantic and geometric data to achieve artifact-free Gaussian Splatting SLAM in dynamic environments.
• GeoCFNet: Geometry-Aware Confidence Field Network for Robot-Assisted Endoscopic Submucosal Dissection — GeoCFNet enhances robot-assisted ESD guidance via geometry-aware confidence field estimation.
• DeGenseGS: Geometrically and Semantically Decoupled Surgical Scene Understanding in 4D Gaussian Splatting — DeGenseGS decouples semantic and geometric modeling in 4D Gaussian Splatting for robust surgical scene understanding.
• UMI-3D: Extending Universal Manipulation Interface from Vision-Limited to 3D Spatial Perception — UMI-3D enhances robot manipulation data collection via LiDAR-centric SLAM and multimodal sensor fusion.
• Pix2Act: Image-Space Manipulation Policies with Equivariant Augmentation — Pix2Act simplifies 3D robotic control to 2D image-space prediction using equivariant augmentation for robust generalization.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.AI updates on arXiv.org.