MakeupMirror: Improving Facial Attribute Preservation in Diffusion Models for Makeup Transfer

2026-06-18 · Source: Artificial Intelligence · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Computer Vision · Depth: Expert, quick

Summary

MakeupMirror is a new diffusion-based model designed to significantly improve facial attribute preservation in makeup transfer, addressing limitations found in prior solutions like Stable-Makeup. Developed for augmented reality and virtual try-on applications, MakeupMirror integrates several technical innovations. These include facial geometry conditioning via ControlNets to maintain fidelity, region-specific makeup transfer control for precise application, and skin tone-based modulation to prevent alterations during cross-subject transfers. It also incorporates a Levenberg-Marquardt Langevin sampler, achieving inference in 0.7s. Experiments on CPM-Real, Makeup Wild, and MakeupSelfies datasets demonstrate MakeupMirror's effectiveness, showing a +60% relative improvement in facial recognition similarity and a -50% reduction in relative skin tone difference compared to Stable-Makeup, alongside a 94% expert acceptance rate for identity preservation.

Key takeaway

For computer vision engineers developing augmented reality or virtual try-on solutions, MakeupMirror offers a significant advancement in makeup transfer realism. You should consider integrating its facial geometry conditioning with ControlNets, region-specific controls, and skin tone modulation to prevent identity and skin tone alterations. This approach can dramatically improve user acceptance and fidelity, making your production-level virtual try-on experiences more convincing and accurate.

Key insights

MakeupMirror enhances diffusion-based makeup transfer by preserving facial identity and skin tone through specific architectural and sampling innovations.

Principles

• Facial fidelity requires geometry-aware conditioning.
• Region-specific control enhances makeup transfer precision.

Method

MakeupMirror integrates ControlNets for geometry, region-specific controls, skin tone modulation, and a Levenberg-Marquardt Langevin sampler into a diffusion model for improved makeup transfer.

In practice

• Use ControlNets for facial feature preservation.
• Implement region-specific controls for precise makeup.
• Apply skin tone modulation in cross-subject transfers.

Topics

• Diffusion Models
• Makeup Transfer
• Facial Attribute Preservation
• Virtual Try-On
• ControlNets
• Augmented Reality

Best for: Research Scientist, AI Product Manager, AI Scientist, Machine Learning Engineer, Computer Vision Engineer

Related on AIssential

• Real2SAM2Real: Generative 3D Caches as Complementary Context for Video Diffusion — Real2SAM2Real uses editable 3D caches from lifting models to provide robust geometric scaffolds for Video Diffusion Models, improving control and consistency.
• Generalized Disguise Makeup Presentation Attack Detection Using an Attention-Guided Patch-Based Framework — A two-phase, attention-guided, patch-based framework effectively detects realistic disguise makeup attacks.
• AI Can Rebuild Blurred Faces, So How Do We Protect People Now? — Traditional blurring is no longer a reliable identity protection method against advanced AI reconstruction tools, necessitating new anonymization strategies.
• FaceCam: Portrait Video Camera Control via Scale-Aware Conditioning — FaceCam uses a scale-aware, face-tailored representation for robust camera control in portrait video generation.
• HairPort: In-context 3D-aware Hair Import and Transfer for Images — HairPort enables robust 3D-aware hairstyle transfer by separating hair removal, enforcing geometric consistency, and leveraging 3D reconstruction.
• LatRef-Diff: Latent and Reference-Guided Diffusion for Facial Attribute Editing and Style Manipulation — LatRef-Diff uses style codes and a novel training strategy for precise, stable facial attribute and style manipulation.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Artificial Intelligence.