PIXLRelight: Controllable Relighting via Intrinsic Conditioning

2026-05-18 · Source: Takara TLDR - Daily AI Papers · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Computer Graphics & Vision · Depth: Expert, medium

Summary

PIXLRelight is a novel feed-forward method for physically controllable single-image relighting, addressing limitations of existing approaches that offer limited control, accumulate errors, or require costly per-image optimization. The core innovation lies in bridging physically based rendering (PBR) and learned image synthesis through a shared intrinsic conditioning. During training, the model is conditioned by decomposing paired multi-illumination photographs into albedo, diffuse shading, and non-diffuse residuals. For inference, this conditioning is derived from a path-traced render of a coarse 3D reconstruction of the input image under user-defined PBR lights. A transformer-based neural renderer then applies the target illumination, preserving fine details via per-pixel affine modulation. PIXLRelight achieves arbitrary PBR-style lighting control, delivers state-of-the-art relighting quality, and processes images in under a tenth of a second.

Key takeaway

For research scientists developing real-time rendering or image editing applications, PIXLRelight offers a robust solution for controllable single-image relighting. You should explore its intrinsic conditioning approach to achieve physically accurate lighting changes without the computational overhead of traditional methods, potentially integrating its transformer-based neural renderer for rapid, high-quality visual effects.

Key insights

PIXLRelight enables fast, high-quality single-image relighting by integrating PBR with learned image synthesis via intrinsic conditioning.

Principles

• Intrinsic conditioning unifies PBR and learned image synthesis.
• Decomposition into albedo, diffuse shading, and residuals is effective.
• Per-pixel affine modulation preserves fine image details.

Method

Decompose multi-illumination photos into intrinsic components for training. At inference, compute conditioning from a path-traced 3D reconstruction under PBR lights, then apply a transformer-based neural renderer with per-pixel affine modulation.

In practice

• Achieves arbitrary PBR-style lighting control.
• Processes images in under 0.1 seconds.
• Code and models are publicly available.

Topics

• PIXLRelight
• Single-Image Relighting
• Physically Based Rendering
• Intrinsic Conditioning
• Neural Rendering

Code references

• alibaba-damo-academy/Lumos-Custom

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, Computer Vision Engineer

Related on AIssential

• PIXLRelight: Controllable Relighting via Intrinsic Conditioning — PIXLRelight enables physically accurate, controllable single-image relighting by bridging PBR and learned synthesis via intrinsic conditioning.
• Continuous Splatting meets Retinex: Continuous Gaussian Splatting and Implicit Reflectance Modeling for Low-Light Image Enhancement — CGS-Retinex enhances low-light images by decoupling illumination and reflectance using continuous Gaussian splatting and implicit neural representation.
• Pixel Cube: Diffusion-based Portrait Video Relighting Through Realistic Lighting Reproduction — Diffusion models, trained on hybrid real/synthetic data from a specialized LED stage, enable photorealistic, temporally consistent portrait video relighting.
• Intrinsic decomposition and editing of 3D Gaussian splats — Intrinsic decomposition for 3D Gaussian splats enables texture editing independent of lighting across viewpoints.
• LUCID: Learning Unified Control for Image Deflaring and Exposure Mastery in Nighttime Photography — LUCID unifies controllable flare mitigation and exposure adjustment for superior nighttime image restoration.
• Multinex: An ultra lightweight AI model advancing low light image enhancement — Multinex uses classical color vision and Retinex principles for efficient, compact low-light image enhancement.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Takara TLDR - Daily AI Papers.