SwinIFS: Landmark Guided Swin Transformer For Identity Preserving Face Super Resolution

2026-07-11 · Source: cs.AI updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Computer Vision · Depth: Expert, extended

Summary

SwinIFS is a novel landmark-guided super-resolution framework designed to reconstruct high-quality facial images from severely degraded low-resolution inputs, preserving identity and fine structural details. It integrates dense Gaussian heatmaps of five key facial landmarks into an 8-channel input representation, guiding the network to focus on semantically important regions. Utilizing a compact Swin Transformer backbone, SwinIFS captures long-range contextual information while maintaining local geometry, enabling the restoration of subtle facial textures and global structural consistency. Extensive experiments on the CelebA benchmark demonstrate its superior perceptual quality, sharper reconstructions, and improved identity retention, particularly at challenging 8× magnification where it achieves a PSNR of 27.97dB and an SSIM of 0.851. SwinIFS also offers an advantageous balance between accuracy and computational efficiency, with an inference time of 0.015 seconds per 128×128 face.

Key takeaway

For Machine Learning Engineers developing face enhancement or surveillance systems, SwinIFS offers a robust solution for high-fidelity super-resolution. You should consider integrating landmark-guided Swin Transformer architectures to achieve superior identity preservation and sharper reconstructions, even from highly degraded inputs at 8× magnification. This approach provides an efficient balance of quality and speed, making it suitable for real-time applications.

Key insights

SwinIFS fuses landmark heatmaps with Swin Transformers for identity-preserving face super-resolution, excelling at extreme upscaling factors.

Principles

• Integrating geometric priors improves FSR robustness.
• Hierarchical attention captures long-range facial dependencies.
• Balancing L1 and perceptual loss enhances realism.

Method

SwinIFS concatenates LR images with five Gaussian landmark heatmaps into an 8-channel input. It uses a Swin Transformer backbone with Residual Swin Transformer Blocks (RSTBs) and PixelShuffle upsampling, trained with L1 and VGG perceptual losses.

In practice

• Use landmark heatmaps for FSR input.
• Employ Swin Transformers for efficient global context.
• Combine L1 and perceptual losses for realism.

Topics

• Face Super-Resolution
• Swin Transformer
• Landmark Guidance
• Identity Preservation
• Image Restoration
• CelebA Dataset

Code references

• Habiba123-stack/SwinIFS

Best for: Research Scientist, AI Scientist, Computer Vision Engineer, Machine Learning Engineer

Related on AIssential

• You Only Landmark Once: Lightweight U-Net Face Super Resolution with YOLO-World Landmark Heatmaps — YOLO-World heatmaps can guide lightweight U-Net face super-resolution without auxiliary training.
• TIGER: Taming Identity, Geometry, and Generative Priors for High-Quality Face Video Restoration — TIGER fuses identity, geometry, and generative priors to restore high-fidelity, identity-consistent face videos efficiently.
• Efficient Transformer-Based Localized Patch Sampling for Choroid Plexus Segmentation in Multiple Sclerosis — A SwinUNETR pipeline with localized patch sampling accurately segments choroid plexus in MS with 99% less computational cost.
• Reference-Based Face Super-Resolution Using the Spatial Transformer — A reference-based face super-resolution model uses a stable spatial transformer for alignment and selective aggregation to achieve state-of-the-art results.
• Reweighting Framewise Attention in Video Transformers for Facial Expression Understanding — Reweighting ViT attention based on frame-level confidence and intra-frame concentration improves subtle facial dynamics understanding.
• Addressing Detail Bottlenecks in Latent Diffusion for RGB-to-SWIR Image Translation — Latent Diffusion Models can overcome detail loss in image translation by enhancing autoencoder and conditioning pathways with learned, high-resolution feature injection.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.AI updates on arXiv.org.