Breaking the Lock-in: Diversifying Text-to-Image Generation via Representation Modulation

2026-06-05 · Source: Computer Vision and Pattern Recognition · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Computer Vision · Depth: Expert, quick

Summary

Recent text-to-image models, built on large-scale Transformer backbones and flow-based objectives, often produce overly similar samples despite strong text-image alignment and high visual quality. Researchers observed that the zero-frequency spatial average (DC) component in intermediate Transformer features rapidly converges across seeds early in generation, causing an "early trajectory lock-in" that limits downstream variation. To address this, they propose DC Attenuation for diVersity Enhancement (DAVE), a training-free, representation-level intervention. DAVE selectively attenuates this DC component in the early generation regime, preserving the sampling pipeline with negligible overhead while improving prompt-consistent diversity and maintaining competitive image quality.

Key takeaway

For Machine Learning Engineers developing text-to-image models who struggle with sample homogeneity, DAVE offers a training-free method to significantly increase output diversity. You should investigate integrating DC Attenuation into your early generation pipeline to achieve more varied results without incurring substantial computational overhead or requiring auxiliary optimization. This approach maintains image quality while breaking early trajectory lock-in.

Key insights

Early convergence of the zero-frequency spatial average (DC) component in Transformer features limits text-to-image generation diversity.

Principles

• DC component convergence causes early trajectory lock-in.
• Attenuating specific feature components enhances diversity.
• Training-free interventions can improve model outputs.

Method

DAVE selectively attenuates the zero-frequency spatial average (DC) component within intermediate Transformer features during the early stages of text-to-image generation to prevent early trajectory lock-in.

In practice

• Implement DAVE for diverse text-to-image outputs.
• Apply DAVE to avoid expensive diversity optimization.
• Integrate DAVE into existing sampling pipelines.

Topics

• Text-to-Image Generation
• Representation Modulation
• Transformer Models
• Image Diversity
• DC Attenuation
• Flow-based Models

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, Computer Vision Engineer

Related on AIssential

• Initialization is Half the Battle: Generating Diverse Images from a Guidance Potential Posterior — Initializing generative models from a guidance potential posterior, rather than standard Gaussian noise, significantly enhances output diversity and quality.
• Extending One-Step Image Generation from Class Labels to Text via Discriminative Text Representation — Highly discriminative text features are crucial for effective one-step text-to-image generation within limited refinement steps.
• Modality Forcing for Scalable Spatial Generation — Modality Forcing enables scalable, joint image-depth generation from sparse data by employing T2I priors with a simple post-training recipe.
• Consistency-Preserving Diverse Video Generation — A new flow-matching framework improves video diversity and temporal consistency using latent-space gradient regulation.
• DSL-LLaDA: Scaling Continuous Denoising to 8B Masked Diffusion LMs — Adapting pretrained masked diffusion LMs with continuous denoising via soft masking resolves few-step generation tradeoffs.
• Rethinking Token Reduction for Diffusion Models via Output-Similarity-Awareness — Output-centric token reduction in Diffusion Models, guided by prior-step output similarity, minimizes recovery error and boosts generation quality.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computer Vision and Pattern Recognition.