Test-Time Noise Guided Adaptation for Realistic Autoregressive Video Generation

2026-07-17 · Source: Computer Vision and Pattern Recognition · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, quick

Summary

Autoregressive video diffusion models, while enabling arbitrarily long video generation and improving computational efficiency by removing future frame conditioning, suffer from error accumulation and drift, leading to "terminal points." To address this, researchers introduce Terminal points Avoidance through Noise Guided Optimization (TANGO). TANGO hypothesizes that for well-modeled future trajectories, the predicted noise distribution should match the forward noising process. At test time, TANGO uses the diffusion model as a critic, predicting one step forward and requiring an isotropic Gaussian noise prediction. Deviations from this expectation guide the search for alternative trajectories, preventing terminal points. This approach achieves a 3.1% absolute improvement on VBench and reduces Fréchet Video Distance by 28.3% on average across 15s videos.

Key takeaway

For computer vision engineers developing autoregressive video generation systems, TANGO offers a method to mitigate error accumulation and "terminal points." By guiding test-time adaptation with noise prediction, your models can achieve more realistic and coherent long video sequences. Consider integrating noise-guided optimization to improve VBench scores and significantly reduce Fréchet Video Distance in your applications.

Key insights

TANGO prevents autoregressive video generation errors by enforcing an isotropic Gaussian noise prior at test time.

Principles

• Well-modeled future trajectories should yield predicted noise distributions matching the forward noising process.

Method

TANGO uses a diffusion model as a critic, predicting one step forward and requiring isotropic Gaussian noise. Deviation from this guides alternative trajectory search.

In practice

• Improve realism in long video generation.
• Reduce Fréchet Video Distance in autoregressive models.

Topics

• Autoregressive Video Generation
• Diffusion Models
• Test-Time Adaptation
• Noise Guidance
• Video Quality Metrics
• VBench

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, Computer Vision Engineer

Related on AIssential

• VPG: Visual Prefix Guidance for Autoregressive Image and Video Generation — VPG is a training-free inference method that improves autoregressive generation by strengthening prefix posterior support.
• Spectrally-Guided Diffusion Noise Schedules — Per-instance noise schedules based on spectral properties improve diffusion model generative quality.
• TetherCache: Stabilizing Autoregressive Long-Form Video Generation with Gated Recall and Trusted Alignment — Stabilizing long-form autoregressive video generation requires selective memory recall and statistical alignment of cached features.
• OPSD-V: On-Policy Self-Distillation for Post-Training Few-Step Autoregressive Video Generators — OPSD-V uses on-policy self-distillation with real long-video context to reduce error accumulation in few-step autoregressive video generators.
• HiAR: Efficient Autoregressive Long Video Generation via Hierarchical Denoising — Conditioning on same-noise-level context in autoregressive diffusion improves temporal consistency and mitigates error propagation.
• SparVAR: Exploring Sparsity in Visual AutoRegressive Modeling for Training-Free Acceleration — SparVAR accelerates Visual AutoRegressive models by exploiting attention sparsity for training-free, high-resolution image generation.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computer Vision and Pattern Recognition.