LearniBridge: Learnable Calibration of Feature Caching for Diffusion Models Acceleration

2026-06-25 · Source: Computer Vision and Pattern Recognition · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, quick

Summary

LearniBridge is a novel learnable calibration mechanism designed to accelerate Diffusion Transformers (DiTs), which are known for their high computational costs in image and video generation. While existing feature caching methods reuse intermediate representations for faster inference, they often suffer significant error accumulation at high acceleration ratios. LearniBridge addresses this by identifying that optimal feature correction updates are characterized by a shared low-rank subspace across diverse prompts. It employs lightweight LoRA updates to bridge multiple timesteps, enabling effective calibration with only 3-5 training samples. Extensive experiments demonstrate LearniBridge achieves substantial acceleration, reaching up to 5.87x on FLUX, 5.75x on HunyuanVideo, and 4.10x on WAN2.1. Furthermore, on WAN2.1, it improves VBench by 1.28% over the previous state-of-the-art at 4.10x acceleration.

Key takeaway

For Machine Learning Engineers optimizing Diffusion Transformer inference, LearniBridge offers a compelling solution to overcome feature caching limitations. If you are struggling with error accumulation at high acceleration ratios, consider implementing LearniBridge's learnable calibration. It enables significant speedups, like 5.87x on FLUX, while improving quality, demonstrated by a 1.28% VBench gain on WAN2.1. Evaluate its lightweight LoRA updates for your DiT models, especially given its minimal 3-5 training sample requirement.

Key insights

LearniBridge accelerates DiTs via learnable LoRA calibration of feature caching, correcting errors with few samples for high acceleration.

Principles

• Optimal feature correction shares a low-rank subspace.
• LoRA updates can bridge multiple timesteps.
• Effective calibration needs only 3-5 training samples.

Method

LearniBridge calibrates feature caching by identifying a shared low-rank subspace for optimal correction. It applies lightweight LoRA updates to bridge multiple timesteps, requiring only 3-5 training samples for effective calibration.

In practice

• Accelerate DiT-based image generation.
• Speed up DiT-based video generation.
• Improve VBench scores at high acceleration.

Topics

• Diffusion Transformers
• Feature Caching
• Model Acceleration
• LoRA
• Image Video Generation

Code references

• Iiiiiiirene/LearniBridge

Best for: AI Engineer, Research Scientist, AI Scientist, Machine Learning Engineer, Computer Vision Engineer

Related on AIssential

• Enhanced Low-Density Region Exploration in Classifier-Guided Diffusion Models Through Modified Reverse Diffusion Sampling — A novel sampling-time modification for classifier-guided diffusion models improves low-density region exploration without retraining.
• TetherCache: Stabilizing Autoregressive Long-Form Video Generation with Gated Recall and Trusted Alignment — Stabilizing long-form autoregressive video generation requires selective memory recall and statistical alignment of cached features.
• RiT: Vanilla Diffusion Transformers Suffice in Representation Space — DINOv2's favorable geometric properties enable vanilla Diffusion Transformers with "x"-prediction to achieve superior, efficient image generation.
• DiffusionBench: On Holistic Evaluation of Diffusion Transformers — ImageNet-centric DiT evaluation is insufficient; T2I performance does not correlate, necessitating holistic benchmarks like DiffusionBench.
• Temporal Difference Learning for Diffusion Models — Temporal difference learning improves diffusion model consistency and sample quality, especially for few-step sampling.
• Diff-SBSR: Learning Multimodal Feature-Enhanced Diffusion Models for Zero-Shot Sketch-Based 3D Shape Retrieval — Diffusion models, enhanced with multimodal features, can effectively perform zero-shot sketch-based 3D shape retrieval.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computer Vision and Pattern Recognition.