SANA-Video 2.0: Hybrid Linear Attention with Attention Residuals for Efficient Video Generation

2026-07-23 · Source: Computer Vision and Pattern Recognition · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Computer Vision · Depth: Expert, quick

Summary

SANA-Video 2.0 is a new hybrid video diffusion transformer, available in 5B and 14B parameter scales, designed for generating high-quality video up to 720p on a single GPU. It achieves quality comparable to full-softmax video DiTs while maintaining the efficient long-sequence scaling of linear attention. The architecture incorporates Hybrid Linear-Softmax Attention, which combines gated linear attention with periodic gated-softmax anchors at a 3:1 ratio to restore full-rank token interactions. Additionally, Block Attention Residuals (AttnRes) propagate refreshed representations by routing completed block summaries into later linear layers, boosting deep-layer effective rank by ~12%. Trained from scratch, SANA-Video 2.0 uses 25% softmax for optimal quality-efficiency. It achieves a VBench score of 84.30 in 13.2s at 480p on a single H100, with its compiled DiT forward pass being 3.2x faster than a full-softmax baseline at 720p/60s. Further Sol-Engine optimization accelerates the 5B pipeline to 13.06s at 720p/5s, making it 120x faster than Wan 2.2-A14B. Overall, this hybrid design recovers softmax-level expressiveness at substantially reduced cost, enabling scalable long, high-resolution video generation.

Key takeaway

For Machine Learning Engineers developing video generation models, SANA-Video 2.0 offers a significant leap in efficiency without sacrificing quality. You can achieve 720p video generation on a single H100 GPU with substantially reduced latency, potentially 120x faster than previous models like Wan 2.2-A14B. Consider adopting hybrid attention architectures and Sol-Engine optimization to scale your video generation capabilities for longer, higher-resolution outputs.

Key insights

SANA-Video 2.0 uses hybrid linear-softmax attention and attention residuals to achieve high-quality, efficient video generation.

Principles

• Hybrid attention balances quality and efficiency.
• Attention residuals boost deep-layer effective rank.
• From-scratch training optimizes hybrid architectures.

Method

SANA-Video 2.0 combines gated linear attention with 25% periodic gated-softmax anchors (3:1 ratio) and Block Attention Residuals to propagate features, then optimizes with Sol-Engine.

In practice

• Generate 720p video on a single H100.
• Achieve 120x speedup over prior models.
• Optimize video generation latency.

Topics

• Video Generation
• Diffusion Transformers
• Hybrid Attention
• Linear Attention
• Attention Residuals
• Sol-Engine Optimization

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, Computer Vision Engineer

Related on AIssential

• VideoMLA: Low-Rank Latent KV Cache for Minute-Scale Autoregressive Video Diffusion — VideoMLA significantly reduces KV cache memory in video diffusion by using low-rank latent attention, improving long-horizon performance and throughput.
• Forget Attention: Importance-Aware Attention Is All You Need — SISA integrates SSM importance directly into attention scores for improved hybrid language model performance.
• SANA-WM: Efficient Minute-Scale World Modeling with Hybrid Linear Diffusion Transformer — SANA-WM offers efficient, high-fidelity minute-scale video generation with precise camera control via hybrid attention and a two-stage pipeline.
• MagicPrompt: Ultra-Lightweight Prompt Tuning for Video Generation — MagicPrompt enables efficient video diffusion model adaptation using minimal parameters and stable dual-space reward optimization.
• Sparse Forcing: Native Trainable Sparse Attention for Real-time Autoregressive Diffusion Video Generation — Sparse Forcing improves long-horizon video generation quality and efficiency by leveraging trainable sparse attention and persistent memory.
• The Sequence Knowledge #878: Beyond Transformer: What We Learned — The Transformer's architectural monoculture is ending, yielding to hybrid models combining attention with linear-scaling alternatives.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computer Vision and Pattern Recognition.