PVRF: All-in-one Adverse Weather Removal via Prior-modulated and Velocity-constrained Rectified Flow

2026-05-15 · Source: cs.CV updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Computer Vision · Depth: Expert, extended

Summary

PVRF is a novel, unified framework designed for all-in-one adverse weather removal (AWR) in real-world images, addressing challenges like heterogeneous degradations and overly smooth restoration results. Developed by researchers from McMaster University and Shanghai Jiao Tong University, PVRF integrates zero-shot soft weather perceptions with velocity-constrained rectified-flow refinement. It features an AWR-specific question answering module (AWR-QA) that leverages frozen vision-language models (VLMs) to estimate soft probabilities of weather types and low-level attribute scores. These perceptions condition restoration networks through Attribute-Modulated Normalization (AMN) and Weather-Weighted Adapters (WWA), providing an anchor estimate for refinement. PVRF then learns a terminal-consistent residual rectified flow with perception-adaptive source perturbation and terminal-consistent velocity parameterization, stabilizing learning near the terminal regime. Extensive experiments demonstrate that PVRF significantly improves both fidelity and perceptual quality over state-of-the-art baselines, showing strong cross-dataset generalization on single and combined degradations.

Key takeaway

For research scientists developing robust image restoration models, PVRF offers a compelling approach to overcome limitations of traditional AWR. You should consider integrating VLM-extracted soft perceptions and terminal-consistent rectified flow into your models to achieve superior fidelity and perceptual quality, especially for handling diverse and combined weather degradations in real-world scenarios.

Key insights

PVRF unifies adverse weather removal by combining VLM-derived soft perceptions with a perception-guided rectified flow for enhanced fidelity and realism.

Principles

• Soft conditioning is superior to hard one-hot encoding for mixed degradations.
• Perception-distortion trade-off requires balancing fidelity and perceptual quality.
• Terminal-consistent velocity parameterization stabilizes rectified flow learning.

Method

PVRF uses a VLM-based AWR-QA module for soft weather and attribute perceptions, then applies AMN and WWA for degradation-aware posterior estimation, followed by a terminal-consistent residual rectified flow with adaptive source perturbation for photo-realistic refinement.

In practice

• Use VLMs for zero-shot, soft weather perception.
• Modulate normalization and adapters with perception priors.
• Employ residual rectified flow for photo-realistic refinement.

Topics

• Adverse Weather Removal
• Rectified Flow
• Vision-Language Models
• Zero-shot Perception
• Degradation-aware Conditioning

Code references

• dongw22/PVRF

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, Computer Vision Engineer

Related on AIssential

• DRIFT: A Residual Flow Adapter for Decoding Continuous Outputs in Vision-Language Models — DRIFT adapts VLMs for precise continuous outputs by refining coarse predictions with a flow matching-based residual generative module.
• SpectralDiT: Timestep-Conditioned Spectral Residual Correction for Flow-Matching DiTs — SpectralDiT enhances Diffusion Transformers by applying timestep-conditioned spectral correction to residual updates, improving FID with minimal overhead.
• Multi-modality Image Fusion under Adverse Weather: Mask-Guided Feature Restoration and Interaction — Mask-guided fusion with pseudo ground truth and attention improves multi-modality image processing under adverse weather.
• Divergence-Suppressing Couplings for Rectified Flow — Trajectory entanglement in Rectified Flow is caused by divergent velocity fields, which can be corrected offline.
• TFTF: Training-Free Targeted Flow for Conditional Sampling — TFTF enables training-free conditional sampling in flow matching by combining importance sampling with stochastic flow and SMC resampling.
• Coreset-Induced Conditional Velocity Flow Matching — CCVFM improves generative flow matching by using a data-informed coreset surrogate instead of isotropic noise.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CV updates on arXiv.org.