SpectralDiT: Timestep-Conditioned Spectral Residual Correction for Flow-Matching DiTs

2026-06-17 · Source: Takara TLDR - Daily AI Papers · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Computer Vision · Depth: Expert, medium

Summary

Jiayu Tian proposes SpectralDiT, a lightweight modification for flow-matching Diffusion Transformers (DiTs) that integrates timestep-conditioned spectral correction into the MLP residual branch. This module operates by decomposing each residual update into low- and high-frequency components on the patch-token grid, then learns a zero-initialized additive gate to initially match the baseline DiT performance. On CIFAR-10 pixel-space generation, SpectralDiT improved the FID score from 20.78 to 19.71 at patch size 1 and reduced the radial Fourier spectrum gap. When scaled to latent diffusion on ImageNet-100, the method achieved an 8.7% relative FID reduction under classifier-free guidance (CFG 2.0), with only 0.6% additional theoretical FLOPs and 1.36% additional parameters. Ablation studies on CIFAR-10 confirmed stable block-specific spectral correction patterns.

Key takeaway

For Machine Learning Engineers optimizing Diffusion Transformers for image generation, consider integrating SpectralDiT's timestep-conditioned spectral correction. This lightweight modification improves FID scores, such as an 8.7% relative reduction on ImageNet-100, with minimal overhead (0.6% FLOPs, 1.36% parameters). You can achieve better generation quality and reduced spectral distortion without significant computational cost, making it a practical enhancement for your flow-matching DiT implementations.

Key insights

SpectralDiT enhances Diffusion Transformers by applying timestep-conditioned spectral correction to residual updates, improving generation quality with minimal overhead.

Principles

• Decomposing residual updates into spectral components can improve DiT performance.
• Timestep-conditioned gates allow adaptive spectral correction.
• Zero-initialization ensures baseline performance matching.

Method

SpectralDiT modifies flow-matching DiTs by adding a timestep-conditioned spectral correction module to the MLP residual branch, decomposing updates into frequency components and learning a zero-initialized additive gate.

In practice

• Apply spectral correction to DiT residual branches.
• Use zero-initialized gates for stable integration.
• Evaluate FID and Fourier spectrum gap.

Topics

• SpectralDiT
• Diffusion Transformers
• Flow Matching
• Spectral Correction
• Image Generation
• FID Score

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, Computer Vision Engineer

Related on AIssential

• Show the Signal, Hide the Noise: Spectral Forcing for Pixel-Space Diffusion — Spectral Forcing improves pixel-space diffusion by explicitly filtering high-frequency noise, optimizing model capacity for signal processing.
• MMDiff: Extending Diffusion Transformers for Multi-Modal Generation — Perceptual information in diffusion transformers is temporally distributed, enabling multi-modal generation via lightweight decoders on frozen backbones.
• Support-Conditioned Flow Matching Is Kernel Smoothing — Cross-attention conditioning in generative models is kernel smoothing, with predictable failure modes and learned corrections.
• Rethinking Token Reduction for Diffusion Models via Output-Similarity-Awareness — Output-centric token reduction in Diffusion Models, guided by prior-step output similarity, minimizes recovery error and boosts generation quality.
• RiT: Vanilla Diffusion Transformers Suffice in Representation Space — DINOv2's favorable geometric properties enable vanilla Diffusion Transformers with "x"-prediction to achieve superior, efficient image generation.
• Spectrally-Guided Diffusion Noise Schedules — Per-instance noise schedules based on spectral properties improve diffusion model generative quality.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Takara TLDR - Daily AI Papers.