RiT: Vanilla Diffusion Transformers Suffice in Representation Space

2026-05-22 · Source: cs.CV updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, extended

Summary

The Representation Image Transformer (RiT) demonstrates that vanilla Diffusion Transformers, when trained with "x"-prediction on frozen DINOv2 features, achieve competitive image generation performance. RiT leverages DINOv2's favorable representation space, which exhibits 7.3× higher effective rank, 35× better covariance conditioning, 11.5× lower excess kurtosis, and 1.7× lower on-manifold interpolation error compared to pixel space, despite similar intrinsic dimensionality (“d̂ ≈ 33). This eliminates the need for complex architectural modifications or Riemannian transport. RiT, augmented with a dimension-aware noise schedule and joint [CLS]-patch modeling, achieves FID 1.45 without guidance and 1.14 with classifier-free guidance on ImageNet 256×256. It uses 19% fewer parameters (676M) than DiT⁄⁄-XL (839M) and enables efficient few-step generation, reaching FID 2.0 in 5 Heun steps and 1.25 in 10 steps with guidance.

Key takeaway

For machine learning engineers developing high-fidelity image generation models, you should consider DINOv2 features as a robust foundation. By employing "x"-prediction with a vanilla Diffusion Transformer on standardized DINOv2 features, you can achieve competitive FID scores with fewer parameters and significantly faster inference steps, avoiding complex architectural additions or Riemannian flow matching. This approach simplifies your generative pipeline and accelerates development.

Key insights

DINOv2's favorable geometric properties enable vanilla Diffusion Transformers with "x"-prediction to achieve superior, efficient image generation.

Principles

• Representation space geometry significantly impacts diffusion model performance.
• "x"-prediction simplifies learning by targeting the data manifold directly.
• High effective rank and well-conditioned covariance improve ODE solvability.

Method

Train a vanilla Diffusion Transformer using "x"-prediction on element-wise standardized DINOv2 features, incorporating a dimension-aware noise schedule and joint [CLS]-patch modeling.

In practice

• Standardize DINOv2 features element-wise before diffusion.
• Employ "x"-prediction for DINOv2-based models.
• Use a dimension-aware noise schedule for high-dimensional tokens.

Topics

• Diffusion Transformers
• DINOv2 Features
• Flow Matching
• Image Generation
• Representation Learning
• x-prediction
• Few-step Generation

Code references

• lezhang7/RiT

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, Computer Vision Engineer

Related on AIssential

• PTL-Diffusion: Manifold-Aware Diffusion with Periodic Terminal Laws — PTL-Diffusion improves manifold data generation by using periodic, structured terminal laws in its forward noising process.
• Improving Visual Representation Alignment Generation with GRPO — VRPO uses adaptive, reward-guided representation alignment to dynamically optimize diffusion transformer training for better fidelity and speed.
• Pixel-Level Residual Diffusion Transformer: Scalable 3D CT Volume Generation — PRDiT uses a two-stage, coarse-to-fine diffusion transformer to generate high-resolution 3D medical volumes directly at voxel-level.
• SEGA: Spectral-Energy Guided Attention for Resolution Extrapolation in Diffusion Transformers — SEGA dynamically scales DiT attention based on latent spatial-frequency, improving high-resolution image generation beyond training data.
• Quotient-Space Diffusion Models — A new diffusion framework simplifies learning by modeling on quotient spaces, outperforming prior symmetry treatments.
• DiffusionBench: On Holistic Evaluation of Diffusion Transformers — ImageNet-centric DiT evaluation is insufficient; T2I performance does not correlate, necessitating holistic benchmarks like DiffusionBench.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CV updates on arXiv.org.