Rethinking Token Reduction for Diffusion Models via Output-Similarity-Awareness

2026-05-21 · Source: Takara TLDR - Daily AI Papers · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, quick

Summary

DiTo introduces a novel token reduction (TR) paradigm for Diffusion Transformers (DiTs) that addresses their quadratic computational complexity. Existing TR methods for DiTs primarily focus on input token similarity, which misaligns with the generative model's objective of minimizing recovery error by reflecting output token similarity. DiTo shifts this focus to output-centric token reduction, leveraging the observation that output token similarity remains consistent across adjacent timesteps. It uses prior-step similarities as a proxy to establish token correspondences at a Matching timestep, which are then reused across multiple subsequent Reduction timesteps. The method optimizes this interleaved scheduling with Pair Match Ratio (PMR)-guided Interval Scheduling and mitigates approximation errors using Frequency-aware Token Matching. Experiments show DiTo outperforms existing TR methods, achieving 1.6-3.9 dB higher PSNR at comparable speedups and a superior Pareto frontier.

Key takeaway

For Machine Learning Engineers optimizing Diffusion Transformers, DiTo offers a significant performance improvement. If you are struggling with the quadratic computational complexity of DiTs, consider implementing DiTo's output-centric token reduction. This approach, which leverages prior-step output similarities and PMR-guided scheduling, can yield 1.6-3.9 dB higher PSNR at similar speedups, enhancing generation quality while maintaining efficiency. Evaluate integrating Frequency-aware Token Matching to mitigate potential approximation errors and blocking artifacts.

Key insights

DiTo improves Diffusion Transformer efficiency by focusing token reduction on output similarity, not just input similarity.

Principles

• Output token similarity is key for generative model efficiency.
• Prior-step similarities can proxy current output similarities.
• Interleaved matching and reduction optimizes token reuse.

Method

DiTo uses prior-step output similarities to establish token correspondences at a Matching timestep, reusing them across multiple Reduction timesteps, optimized by PMR-guided Interval Scheduling and Frequency-aware Token Matching.

In practice

• Apply output-centric token reduction in DiTs.
• Implement PMR-guided scheduling for optimal matching frequency.
• Use frequency-aware matching to reduce artifacts.

Topics

• Diffusion Transformers
• Token Reduction
• Image Generation
• Computational Complexity
• PSNR
• DiTo

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, Computer Vision Engineer

Related on AIssential

• Rethinking Token Reduction for Diffusion Models via Output-Similarity-Awareness — Output-centric token reduction in Diffusion Models, guided by prior-step output similarity, minimizes recovery error and boosts generation quality.
• DODO: Discrete OCR Diffusion Models — DODO uses block discrete diffusion to achieve faster, accurate OCR by enabling parallel token generation while maintaining structural integrity.
• Balancing Image Compression and Generation with Bootstrapped Tokenization — SelfBootTok decomposes image tokens into global and local groups, predicting local details from global tokens for efficient generation and scalable training.
• DiffMath: Symbol- and Graph-Aware Latent Diffusion Transformer for Handwritten Mathematical Expression Generation — DiffMath generates HMEG without explicit spatial supervision by using LaTeX structure and a structured latent diffusion model.
• DiffusionBench: On Holistic Evaluation of Diffusion Transformers — ImageNet-centric DiT evaluation is insufficient; T2I performance does not correlate, necessitating holistic benchmarks like DiffusionBench.
• Variable-Length Tokenization via Learnable Global Merging for Diffusion Transformers — Merging similar tokens enables variable-length tokenization while preserving cross-length representation alignment in Diffusion Transformers.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Takara TLDR - Daily AI Papers.