LongStream: Long-Sequence Streaming Autoregressive Visual Geometry

2026-02-13 · Source: Computer Vision and Pattern Recognition · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems · Depth: Expert, quick

Summary

LongStream is a novel gauge-decoupled streaming visual geometry model designed for stable, metric-scale 3D scene reconstruction over thousands of frames. Existing autoregressive models struggle with long sequences due to first-frame pose anchoring, which causes attention decay, scale drift, and extrapolation errors. LongStream addresses these issues by predicting keyframe-relative poses, transforming long-range extrapolation into a constant-difficulty local task. It also incorporates orthogonal scale learning to disentangle geometry from scale estimation, effectively suppressing drift. Furthermore, the model tackles Transformer cache problems like attention-sink reliance and KV-cache contamination through cache-consistent training and periodic cache refresh, which prevents attention degradation and reduces the training-inference gap. LongStream achieves state-of-the-art performance, reconstructing kilometer-scale sequences at 18 FPS.

Key takeaway

For research scientists developing streaming 3D reconstruction systems, LongStream's architectural innovations offer a robust solution to long-sequence instability. You should consider adopting keyframe-relative pose prediction and orthogonal scale learning to mitigate drift, alongside cache-consistent training and periodic cache refresh for Transformer-based models to achieve stable, metric-scale performance over extended sequences.

Key insights

LongStream enables stable, metric-scale 3D reconstruction for ultra-long sequences by decoupling gauge and optimizing Transformer caches.

Principles

• Anchor poses to keyframes, not the first frame.
• Disentangle geometry from scale estimation.
• Refresh Transformer KV-caches periodically.

Method

LongStream predicts keyframe-relative poses, employs orthogonal scale learning, and uses cache-consistent training with periodic cache refresh to manage Transformer attention over long sequences.

In practice

• Use keyframe-relative pose prediction for long sequences.
• Implement orthogonal scale learning to reduce drift.
• Apply cache-consistent training for Transformer stability.

Topics

• 3D Reconstruction
• Streaming Visual Geometry
• Autoregressive Models
• Transformer Architectures
• LongStream

Best for: Research Scientist, AI Researcher, AI Scientist, Computer Vision Engineer

Related on AIssential

• StreamCacheVGGT: Streaming Visual Geometry Transformers with Robust Scoring and Hybrid Cache Compression — StreamCacheVGGT enhances 3D geometry reconstruction from video by robustly scoring and compressing cache tokens.
• Stream3D: Sequential Multi-View 3D Generation via Evidential Memory — Stream3D uses an evidential memory and evidence score mechanism to enable consistent 3D generation from sequential multi-view inputs without retraining existing models.
• LiveStre4m: Feed-Forward Live Streaming of Novel Views from Unposed Multi-View Video — LiveStre4m enables real-time novel view synthesis from unposed multi-view video using a feed-forward, calibration-free approach.
• Stream3D-VLM: Online 3D Spatial Understanding with Incremental Geometry Priors — Stream3D-VLM enables real-time 3D spatial understanding from streaming video by integrating geometry priors and adaptive token compression.
• VGGT-$Ω$ — Feed-forward reconstruction models scale predictably with data and model size, offering geometry-aware features.
• TetherCache: Stabilizing Autoregressive Long-Form Video Generation with Gated Recall and Trusted Alignment — TetherCache stabilizes long-form autoregressive video generation by intelligently managing cache and aligning recalled memory to prevent temporal drift.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computer Vision and Pattern Recognition.