Veda: Scalable Video Diffusion via Distilled Sparse Attention

2026-05-28 · Source: Computer Vision and Pattern Recognition · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Computer Vision · Depth: Expert, quick

Summary

Veda introduces a distilled sparse attention framework designed to scale video diffusion transformers for high-resolution, long video generation. It addresses the quadratic cost of self-attention and the degradation issues of existing sparse methods by demonstrating that generation quality is tied to how well the sparse mask aligns with the tile-wise geometry of full attention, rather than just the sparsity ratio. Veda formulates tile selection as an explicit reconstruction problem, integrating statistics-aware tile scoring with head-aware tiling to enable aggressive sparsity. A hardware-efficient tile-skipping kernel translates this theoretical sparsity into practical wall-clock speedups. Experiments on large video diffusion models like Waver and Wan2.1 show substantial acceleration without quality degradation. For 720P 10-second videos on Waver-T2V-12B, Veda achieves a 5.1x end-to-end speedup and a 10.5x self-attention speedup, reducing attention overhead from 92% to 50%.

Key takeaway

For Machine Learning Engineers developing high-resolution video diffusion models, Veda offers a critical solution to the quadratic cost of self-attention. You can achieve substantial speedups, like 5.1x end-to-end and 10.5x for self-attention, without compromising generation quality. Consider integrating Veda's distilled sparse attention to scale your models efficiently, especially for longer or higher-resolution video outputs.

Key insights

Generation quality in sparse attention depends on mask alignment with full attention's tile-wise geometry, not just sparsity ratio.

Principles

• Sparse attention quality hinges on mask-to-tile alignment.
• Aggressive sparsity is achievable with proper alignment.
• Speedup gains increase with sequence length.

Method

Veda formulates tile selection as an explicit reconstruction problem from full attention, integrating statistics-aware tile scoring with head-aware tiling to reduce estimation error and structural mismatch.

In practice

• Generate 720P 10-second videos faster.
• Accelerate Waver and Wan2.1 models.
• Reduce attention overhead significantly.

Topics

• Video Diffusion
• Sparse Attention
• Diffusion Transformers
• Model Acceleration
• Waver
• Wan2.1

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, Computer Vision Engineer

Related on AIssential

• SEGA: Spectral-Energy Guided Attention for Resolution Extrapolation in Diffusion Transformers — SEGA dynamically scales DiT attention based on latent spatial-frequency, improving high-resolution image generation beyond training data.
• SparVAR: Exploring Sparsity in Visual AutoRegressive Modeling for Training-Free Acceleration — SparVAR accelerates Visual AutoRegressive models by exploiting attention sparsity for training-free, high-resolution image generation.
• Building Generative Image & Video models at Scale - Sander Dieleman (Veo and Nano Banana) — Diffusion models generate high-quality audiovisual data through iterative denoising of learned compressed latent representations.
• HiAR: Efficient Autoregressive Long Video Generation via Hierarchical Denoising — Conditioning on same-noise-level context in autoregressive diffusion improves temporal consistency and mitigates error propagation.
• Variance Reduction for Expectations with Diffusion Teachers — CARV reduces Monte Carlo variance in frozen diffusion teacher gradients by hierarchically amortizing expensive computations.
• RiT: Vanilla Diffusion Transformers Suffice in Representation Space — DINOv2's favorable geometric properties enable vanilla Diffusion Transformers with "x"-prediction to achieve superior, efficient image generation.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computer Vision and Pattern Recognition.