Rarity-Aware Discrete Diffusion with Spatially Consistent Decoding for Photo-Realistic Image Super-Resolution

2026-07-23 · Source: cs.CV updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Computer Vision · Depth: Expert, extended

Summary

DiMOO-SR introduces a rarity-aware discrete diffusion framework for photo-realistic image Super-Resolution (SR), addressing key limitations of existing methods. While continuous diffusion models dominate SR, they struggle with direct token-based scaling, and autoregressive models, though token-based, are slow. DiMOO-SR tackles two specific challenges in discrete diffusion for SR: the long-tailed distribution of visual tokens, which under-represents critical textures, and spatially inconsistent parallel decoding, which can introduce artifacts. The framework integrates Inverse Frequency Sampling (IFS) during training to prioritize rare, information-rich tokens, and Spatial Consistency Ranking (SCR) during inference to enhance structural coherence by refining token confidence based on local neighborhood agreement. Experiments on RealSR and DRealSR benchmarks demonstrate DiMOO-SR's competitive perceptual quality, achieving a FID of 123.78 and DISTS of 0.2110 on RealSR, and restoring a 512x512 image in 1.63 seconds with only 5 parallel decoding steps.

Key takeaway

For Machine Learning Engineers developing photo-realistic image Super-Resolution systems, you should consider discrete diffusion models like DiMOO-SR. This framework offers a robust solution for achieving high perceptual quality and efficiency by explicitly addressing the long-tailed distribution of visual tokens and spatial inconsistencies in parallel decoding. Your projects can benefit from its ability to recover fine textures and structural coherence in just 5 decoding steps, making it a strong alternative to slower autoregressive or less direct continuous diffusion approaches.

Key insights

Discrete diffusion models can achieve photo-realistic SR by explicitly managing token rarity and spatial consistency.

Principles

• Rare visual tokens encode high-frequency, perceptually critical details.
• Uniform masking over-emphasizes common, low-information visual tokens.
• Local neighborhood agreement enhances structural coherence in parallel decoding.

Method

DiMOO-SR employs Inverse Frequency Sampling (IFS) to rebalance training towards rare tokens and Spatial Consistency Ranking (SCR) to refine inference confidence using local spatial consensus.

In practice

• Implement importance sampling for rare visual tokens during training.
• Apply reliability-gated confidence refinement in parallel decoding.
• Combine uniform and importance sampling for stable optimization.

Topics

• Discrete Diffusion Models
• Image Super-Resolution
• Inverse Frequency Sampling
• Spatial Consistency Ranking
• Multimodal Generative AI
• Visual Tokenization

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, Computer Vision Engineer

Related on AIssential

• Decoupling Cross-Modality Manifold Discrepancy: Leveraging Visible Diffusion Priors for Infrared Super-Resolution — A dual-path diffusion framework, Shift-IISR, improves infrared image super-resolution consistency while preserving generative model capacity.
• DODO: Discrete OCR Diffusion Models — DODO uses block discrete diffusion to achieve faster, accurate OCR by enabling parallel token generation while maintaining structural integrity.
• DAV-GSWT: Diffusion-Active-View Sampling for Data-Efficient Gaussian Splatting Wang Tiles — DAV-GSWT synthesizes expansive 3D Gaussian Splatting Wang Tiles from sparse data using active view sampling and diffusion priors.
• Toward a mechanistic understanding of inference in visual cortex and diffusion models — A V1-inspired sparse coding model acts as a minimal diffusion model, revealing mechanisms for perceptual inference and novel image generation.
• Efficient Difficulty-Aware Dynamic Routing for Diffusion-Based Real-World Image Super-Resolution — The Difficulty-aware Dynamic Routing (DDR) strategy optimizes diffusion-based image super-resolution by adaptively processing images based on their restoration difficulty.
• Exploiting Semantic and Pixel Representations for Ultra-Low Bitrate Image Compression — Ultra-low bitrate image compression can achieve pixel-level fidelity by combining semantic and pixel representations in a diffusion model.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CV updates on arXiv.org.