Track2Map: Online Deformable SLAM with Motion-Aware Pose Optimization in Robotic Surgery

2026-07-09 · Source: Takara TLDR - Daily AI Papers · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems, Medical Devices & Health Technology · Depth: Expert, medium

Summary

Track2Map is an online 3D Gaussian Splatting pipeline designed for robot-assisted minimally invasive surgery (RAMIS) that jointly optimizes camera trajectory and deformable 3D scene representation directly from surgical video. This system addresses the limitations of existing offline pipelines, which often rely on accurate camera trajectory priors that may be missing or noisy. Functioning as a Simultaneous Localization and Mapping (SLAM) method, Track2Map achieves robust 3D reconstructions even without reliable prior camera data. It incorporates a track-anchored deformation initialization using dense 2D point tracks to stabilize optimization during tissue motion and ambiguous visual cues. Furthermore, it utilizes track statistics to differentiate camera motion from scene deformation, detecting static camera periods and mitigating drift during incremental mapping. Experiments on the StereoMIS dataset demonstrate Track2Map's superior reconstruction quality and camera trajectory accuracy compared to both competing SLAM and non-SLAM methods.

Key takeaway

For Robotics Engineers developing surgical systems, if you require robust 3D reconstruction and camera localization in dynamic environments without perfect camera priors, Track2Map offers a viable online SLAM solution. You should consider integrating its joint optimization of camera trajectory and deformable scene representation. This approach can enhance reconstruction quality and trajectory accuracy, especially when dealing with tissue motion and noisy input, making your systems more reliable for robot-assisted minimally invasive surgery.

Key insights

Track2Map enables robust online deformable 3D reconstruction and camera localization in RAMIS without relying on accurate trajectory priors.

Principles

• Joint optimization of camera trajectory and scene representation improves robustness.
• Track-anchored deformation initialization stabilizes optimization in dynamic scenes.
• Disentangling camera motion from scene deformation reduces drift.

Method

Track2Map jointly optimizes camera trajectory and 3D deformable scene representation from surgical video. It uses track-anchored deformation initialization and track statistics to detect static camera periods and reduce drift.

In practice

• Apply 3D Gaussian Splatting for online deformable SLAM.
• Use 2D point tracks for deformation initialization.
• Implement track statistics to separate motion and deformation.

Topics

• Robotic Surgery
• Deformable SLAM
• 3D Gaussian Splatting
• Camera Trajectory Optimization
• Surgical Video Analysis
• Minimally Invasive Surgery

Best for: AI Scientist, Robotics Engineer, Research Scientist

Related on AIssential

• Track2Map: Online Deformable SLAM with Motion-Aware Pose Optimization in Robotic Surgery — Track2Map robustly performs online deformable SLAM in surgical video by disentangling camera motion from tissue deformation.
• Track2Map: Online Deformable SLAM with Motion-Aware Pose Optimization in Robotic Surgery — Track2Map enables robust online deformable 3D reconstruction in robotic surgery by jointly optimizing camera pose and scene deformation.
• DL-SLAM: Enabling High-Fidelity Gaussian Splatting SLAM in Dynamic Environments based on Dual-Level Probability — DL-SLAM uses a dual-level probabilistic framework combining semantic and geometric data to achieve artifact-free Gaussian Splatting SLAM in dynamic environments.
• DeGenseGS: Geometrically and Semantically Decoupled Surgical Scene Understanding in 4D Gaussian Splatting — DeGenseGS decouples semantic and geometric modeling in 4D Gaussian Splatting for robust surgical scene understanding.
• GLAM-SLAM: Real-time Gaussian Large-scale Mapping via Flow Densification and Spatial Decomposition — GLAM-SLAM enables real-time, large-scale 3D Gaussian Splatting SLAM by decoupling tracking and mapping with spatial decomposition and flow densification.
• Track2View: 4D-Consistent Camera-Controlled Video Generation via Paired 3D Point Tracks — Track2View uses paired 3D point tracks to enable 4D-consistent, camera-controlled video re-rendering with explicit spatiotemporal correspondences.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Takara TLDR - Daily AI Papers.