PIXLRelight: Controllable Relighting via Intrinsic Conditioning

2026-05-18 · Source: Computer Vision and Pattern Recognition · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Computer Vision, Computer Graphics · Depth: Expert, quick

Summary

PIXLRelight is a novel feed-forward method for physically controllable single-image relighting, addressing limitations of existing techniques that offer restricted lighting control or incur high computational costs. This approach integrates physically based rendering (PBR) with learned image synthesis by utilizing a shared intrinsic conditioning derived from either real photographs or PBR renders. During training, multi-illumination photographs are decomposed into albedo, diffuse shading, and non-diffuse residuals to condition the model. For inference, this conditioning is computed from a path-traced render of a coarse 3D reconstruction of the input image under user-defined PBR lights. A transformer-based neural renderer then applies the target illumination, maintaining fine image details via per-pixel affine modulation. PIXLRelight provides arbitrary PBR-style lighting control, delivers high-quality relighting, and processes each image in under 0.1 seconds.

Key takeaway

For computer vision engineers developing real-time rendering or image editing applications, you should evaluate PIXLRelight for its ability to provide physically accurate, controllable relighting at high speeds. Its integration of PBR and neural rendering offers a robust solution for applying arbitrary lighting conditions to single images, potentially streamlining workflows that currently rely on costly per-image optimization or limited control methods. Consider its open-source availability for immediate prototyping.

Key insights

PIXLRelight enables physically accurate, controllable single-image relighting by bridging PBR and learned synthesis via intrinsic conditioning.

Principles

• Decompose images into intrinsic components for robust conditioning.
• Combine PBR and neural rendering for precise light control.

Method

Decompose multi-illumination photos into albedo, diffuse shading, and non-diffuse residuals for training. At inference, compute conditioning from a path-traced 3D reconstruction under PBR lights, then apply with a transformer-based neural renderer.

In practice

• Achieve arbitrary PBR-style lighting control.
• Relight single images in under 0.1 seconds.

Topics

• PIXLRelight
• Single-Image Relighting
• Physically Based Rendering
• Intrinsic Conditioning
• Neural Rendering

Best for: Research Scientist, AI Scientist, Computer Vision Engineer, Machine Learning Engineer

Related on AIssential

• PIXLRelight: Controllable Relighting via Intrinsic Conditioning — PIXLRelight enables fast, high-quality single-image relighting by integrating PBR with learned image synthesis via intrinsic conditioning.
• Continuous Splatting meets Retinex: Continuous Gaussian Splatting and Implicit Reflectance Modeling for Low-Light Image Enhancement — CGS-Retinex enhances low-light images by decoupling illumination and reflectance using continuous Gaussian splatting and implicit neural representation.
• Pixel Cube: Diffusion-based Portrait Video Relighting Through Realistic Lighting Reproduction — Diffusion models, trained on hybrid real/synthetic data from a specialized LED stage, enable photorealistic, temporally consistent portrait video relighting.
• TokenLight: Precise Lighting Control in Images using Attribute Tokens — Attribute tokens enable precise, continuous control over multiple illumination factors in image relighting.
• PhysicEdit: Teaching Image Editing Models to Respect Physics — PhysicEdit improves image editing realism by modeling physical state transitions using video data and dual-thinking mechanisms.
• Multinex: An ultra lightweight AI model advancing low light image enhancement — Multinex uses classical color vision and Retinex principles for efficient, compact low-light image enhancement.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computer Vision and Pattern Recognition.