Image Restoration via Diffusion Models with Dynamic Resolution

2026-05-15 · Source: cs.CV updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Computer Vision · Depth: Expert, long

Summary

This work introduces SubDPS and SubDAPS, novel methods that apply dynamic resolution Diffusion Models (DMs) to general image restoration tasks, addressing the computational inefficiency of existing pixel-space and latent DM approaches. While pixel-space DMs incur high computational overhead due to high-dimensional operations, latent DMs, despite using compressed latent spaces, suffer from repeated encoder-decoder inference, often leading to inferior runtime performance. The proposed methods project data into lower-dimensional subspaces using dynamic resolution DMs, fine-tuning pre-trained DMs for dynamic resolution priors. An enhanced variant, SubDAPS++, is also introduced, which modifies noise injection, incorporates a corrector step without additional model inference, and replaces Langevin dynamics with a conjugate gradient method to further boost efficiency and quality. Empirical evaluations across diverse image datasets and restoration tasks demonstrate that these methods generally outperform recent DM-based approaches in both reconstruction fidelity and sampling efficiency.

Key takeaway

For research scientists developing image restoration solutions, you should consider integrating dynamic resolution Diffusion Models into your workflows. This approach significantly reduces computational overhead compared to traditional pixel-space or latent DM methods, offering superior reconstruction fidelity and sampling efficiency. Explore adapting existing pixel-space methods like DPS and DAPS to this framework, and investigate enhancements such as modified noise injection and conjugate gradient methods to further optimize performance.

Key insights

Dynamic resolution DMs accelerate image restoration by operating in lower-dimensional subspaces, avoiding VAE overhead.

Principles

• Early sampling stages contain computational redundancy.
• Dynamic resolution improves global structure recovery.
• Conjugate gradient method enhances reconstruction speed.

Method

Fine-tune pre-trained DMs for dynamic resolution priors, adapt pixel-space methods (DPS, DAPS) to this framework, and enhance with modified noise injection, a corrector step, and conjugate gradient method.

In practice

• Use dynamic resolution DMs for faster image restoration.
• Implement a corrector step for fidelity improvement.
• Replace Langevin dynamics with conjugate gradient.

Topics

• Image Restoration
• Diffusion Models
• Dynamic Resolution DMs
• Computational Efficiency
• SubDAPS++

Code references

• StarNextDay/SubDAPS

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, Computer Vision Engineer

Related on AIssential

• Estimation of instrument and noise parameters for inverse problem based on prior diffusion model — A Gibbs-Diffusion Posterior Sampling (G-DPS) based method efficiently estimates observation parameters and images in inverse problems with diffusion priors.
• IG-Diff: Complex Night Scene Restoration with Illumination-Guided Diffusion Model — Illumination-guided diffusion models effectively restore complex night scenes by integrating illumination priors and synthetic data.
• Beyond MMSE: Enhancing PnP Restoration with ProxiMAP — ProxiMAP enhances PnP image restoration by aligning denoiser noise schedules to prevent "cartoon-like" artifacts from inexact learned scores.
• Spectral Progressive Diffusion for Efficient Image and Video Generation — Spectral Progressive Diffusion accelerates image/video generation by progressively increasing resolution based on frequency domain content.
• TOC-SR: Task-Optimal Compact diffusion for Image Super Resolution — TOC-SR creates efficient one-step super-resolution models via compact diffusion backbone discovery and distillation.
• Adapting Diffusion Language Models for Lossless Pixel-Level Image Transmission — DDM-SSCC adapts diffusion language models for lossless pixel-level image transmission, achieving superior exact recovery via novel coding and denoising strategies.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CV updates on arXiv.org.