Planning Generated Images In Stages

2026-05-29 · Source: The Batch | DeepLearning.AI | AI News & Insights - www.deeplearning.ai · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, short

Summary

Researchers from Meta, University of California San Diego, Worcester Polytechnic Institute, and Northwestern University introduced a fine-tuning method for text-to-image generators that breaks image composition into discrete stages. Unlike typical diffusion or flow-matching models that compose images at once, this approach trains a model to plan, generate elements, check against the prompt, and correct iteratively. Starting with BAGEL-7B, the method fine-tuned the model across plan, sketch, inspect, and refine stages using datasets generated by GPT-4o. This improved BAGEL-7B's GenEval score from 77 percent to 83 percent with 62,000 examples and 131 flow-matching steps, outperforming PARM's 77 percent with 688,000 examples and 1,000 steps. It also raised BAGEL's WISE score from 0.7 to 0.76, enhancing realism and prompt consistency, particularly for spatial relationships and object attributes.

Key takeaway

For AI Engineers developing text-to-image systems, consider adopting a staged generation and refinement process. This approach significantly improves accuracy in spatial relationships and object attributes, as demonstrated by BAGEL-7B's GenEval score rising from 77 percent to 83 percent with fewer training examples and steps than alternative methods. You should explore fine-tuning existing models with similar iterative feedback loops to enhance prompt adherence and realism, moving beyond simple data scaling.

Key insights

Staged image generation with iterative checking and refinement significantly improves prompt adherence and output quality.

Principles

• Decompose complex generation into discrete stages
• Implement iterative inspection and correction loops
• Train models on staged composition data

Method

Fine-tune a multimodal model like BAGEL-7B to cycle through Plan, Sketch, Inspect, and Refine stages, using GPT-4o-generated datasets for each step to build and correct image compositions.

In practice

• Apply staged generation to improve spatial accuracy
• Use iterative feedback for object attribute correction
• Fine-tune existing models with compositional datasets

Topics

• Text-to-Image Generation
• Diffusion Models
• Multimodal AI
• Iterative Refinement
• Fine-tuning
• Image Composition
• Prompt Adherence

Best for: Computer Vision Engineer, Research Scientist, AI Scientist, Machine Learning Engineer, AI Engineer

Related on AIssential

• Remaking Old Computer Graphics With AI Image Generation — AI image generation can remake old graphics, but requires iterative prompting and tool-specific techniques.
• Can text *finally* make robots dance exactly how we want them to? — Scaling motion generation models with high-quality data enables nuanced instruction-following capabilities.
• Safe Autoregressive Image Generation with Iterative Self-Improving Codebooks — Unified multimodal models can self-identify and eliminate unsafe image generations through iterative codebook refinement.
• TEST: Text to Image AI Generator for Technical Stuff — AI image generators vary significantly in their ability to render complex technical infographs from identical text prompts.
• A Visual Guide to DiffusionGemma — DiffusionGemma uses iterative diffusion to generate text, optimizing single-user latency by processing 256 tokens concurrently.
• Foundation Models for Automatic CAD Generation — Integrating foundation models with iterative, geometry-aware refinement significantly advances automatic parametric 3D CAD generation.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by The Batch | DeepLearning.AI | AI News & Insights - www.deeplearning.ai.