FADRA: Frequency-Aware Diffusion with Residual Adaptation for Video Face Restoration

2026-07-07 · Source: Takara TLDR - Daily AI Papers · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Computer Vision · Depth: Expert, medium

Summary

FADRA is a frequency-aware diffusion framework with iterative residual adaptation specifically tailored for Video Face Restoration (VFR). It addresses the challenge of balancing spatial fidelity and temporal coherence in severely degraded video sequences. FADRA utilizes a pre-trained text-to-video diffusion model, integrating lightweight LoRA adapters and a Low-Quality (LQ) Pixel-Alignment Feature Fusion module to adapt the generative prior for VFR. It further introduces a Repeated Residual Adaptation Head (RRAH) for step-wise residual refinement, guided by LQ latent input and velocity prediction at each flow-matching step. A Frequency-Aware Loss is also incorporated to supervise multiple spectral bands, emphasizing visually sensitive frequency components crucial for perceptual quality and temporal stability. Experiments demonstrate FADRA recovers better facial structures and produces more temporally consistent videos than existing methods, improving both quantitative metrics and visual perception.

Key takeaway

For Machine Learning Engineers developing video face restoration solutions, FADRA offers a robust approach to improve both spatial fidelity and temporal consistency. You should consider integrating frequency-aware loss and iterative residual adaptation into your diffusion models. This framework helps recover fine facial details and reduces temporal jittering, leading to superior visual quality in degraded video sequences.

Key insights

FADRA enhances video face restoration by combining diffusion models with frequency-aware and residual adaptation for temporal consistency.

Principles

• Pre-trained diffusion models can be adapted for VFR.
• Frequency-aware loss improves structural integrity.
• Iterative residual refinement enhances detail recovery.

Method

FADRA adapts a text-to-video diffusion model using LoRA and LQ Pixel-Alignment. It then applies a Repeated Residual Adaptation Head for LQ-guided, step-wise residual refinement, supervised by a Frequency-Aware Loss across spectral bands.

In practice

• Recover high-quality facial details from degraded videos.
• Improve temporal consistency in video restoration.
• Enhance perceptual quality of restored faces.

Topics

• Video Face Restoration
• Diffusion Models
• LoRA Adapters
• Frequency-Aware Loss
• Temporal Consistency
• Residual Adaptation

Code references

• wangzhiyaoo/SVFR
• fudan-generative-vision/DicFace
• SYSUzrc/FDG-Diff

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, Computer Vision Engineer

Related on AIssential

• DVFace: Spatio-Temporal Dual-Prior Diffusion for Video Face Restoration — DVFace uses a one-step, dual-prior diffusion framework for efficient, high-quality video face restoration.
• Diff-SBSR: Learning Multimodal Feature-Enhanced Diffusion Models for Zero-Shot Sketch-Based 3D Shape Retrieval — Diffusion models, enhanced with multimodal features, can effectively perform zero-shot sketch-based 3D shape retrieval.
• RGFVR: Reference-Guided Face Video Restoration with Flow Matching — RGFVR uses reference-guided, bimodal identity conditioning and a two-stage training to restore degraded face videos with identity preservation.
• Plug-and-Play image restoration with Stochastic deNOising REgularization — SNORE improves PnP image restoration by applying denoisers to appropriately noisy images, enhancing performance and theoretical guarantees.
• LatRef-Diff: Latent and Reference-Guided Diffusion for Facial Attribute Editing and Style Manipulation — LatRef-Diff uses style codes and a novel modulation module for precise facial attribute and style manipulation.
• IG-Diff: Complex Night Scene Restoration with Illumination-Guided Diffusion Model — Illumination-guided diffusion models effectively restore complex night scenes by integrating illumination priors and synthetic data.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Takara TLDR - Daily AI Papers.