Freqformer: Image-Demoir\'eing Transformer via Effective Frequency Decomposition

2026-07-08 · Source: cs.CV updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Computer Vision · Depth: Expert, extended

Summary

Freqformer is a novel Transformer-based framework for image demoiréing, addressing the complex interplay of texture corruption and color distortions caused by moiré patterns. Developed by researchers from Shanghai Jiao Tong and Harvard Universities, it employs an effective frequency decomposition to separate moiré into high-frequency textures and low-frequency color distortions. A dual-branch architecture, with crop-based training for high-frequency and resize-based training for low-frequency components, processes these distinct characteristics. Freqformer also introduces a learnable Frequency Composition Transform (FCT) for adaptive output fusion and a Spatial-Aware Channel Attention (SA-CA) module for efficient feature refinement. Extensive experiments on FHDMi and UHDM benchmarks demonstrate Freqformer achieves state-of-the-art performance with a compact model size of 6.065M parameters and 2.49 TFLOPS for 4K images.

Key takeaway

For Machine Learning Engineers developing image processing solutions, Freqformer offers a highly efficient and effective approach to image demoiréing. Its novel frequency decomposition and dual-branch architecture, combined with a learnable Frequency Composition Transform, enable superior artifact removal and color restoration, even for 4K images. You should consider integrating this compact 6.065M parameter model for high-resolution applications or edge device deployment where computational efficiency is critical.

Key insights

Freqformer disentangles moiré patterns into distinct frequency components for targeted, efficient image restoration.

Principles

• Moiré patterns separate into high-frequency textures and low-frequency color distortions.
• High-frequency components benefit from crop-based training due to spatial locality.
• Low-frequency components are scale-robust, enabling resize-based training.

Method

Freqformer uses a two-stage training: independent high/low-frequency branch training, followed by joint fine-tuning with a learnable FCT. It employs a recursive convolution-based frequency decomposition.

In practice

• Deploy on edge devices due to compact model size (6.065M parameters).
• Process ultra-high-resolution images (e.g., 4K) efficiently (2.49 TFLOPS).
• Improve demoiréing by separating texture and color artifacts.

Topics

• Image Demoiréing
• Transformer Networks
• Frequency Decomposition
• Dual-Branch Architecture
• Spatial-Aware Channel Attention
• High-Resolution Imaging

Code references

• xyLiu339/Freqformer

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, Computer Vision Engineer

Related on AIssential

• FADRA: Frequency-Aware Diffusion with Residual Adaptation for Video Face Restoration — FADRA enhances video face restoration by adapting a text-to-video diffusion model with iterative residual refinement and frequency-aware loss.
• FrequencyFormer: A Co-Designed Sensor-to-Processor Pipeline for Frequency-Domain Vision Transformer Inference — Frequency-domain tokenization at the sensor-edge dramatically reduces data movement and energy for Vision Transformer inference, enabling efficient deployment.
• The Third Challenge on Image Denoising at NTIRE 2026: Methods and Results — The NTIRE 2026 challenge benchmarks unconstrained neural architectures for high-noise image denoising, prioritizing PSNR performance.
• FSDC-DETR: A Frequency-Spatial Domain Collaborative DETR for Small Object Detection — FSDC-DETR enhances small object detection by explicitly integrating frequency and spatial domain modeling throughout the detection pipeline.
• Plug-and-Play image restoration with Stochastic deNOising REgularization — SNORE improves PnP image restoration by applying denoisers to appropriately noisy images, enhancing performance and theoretical guarantees.
• FreqLite: A Lightweight Frequency-Decomposed Linear Model with Adaptive Reversible Normalization for Robust Long-Term Time-Series Forecasting — FreqLite is a lightweight, frequency-decomposed linear model with adaptive normalization for robust, efficient long-term time-series forecasting.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CV updates on arXiv.org.