TILDE: TILt-based Distributional Erasure for Concept Unlearning

2026-07-08 · Source: cs.CV updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Generative AI & Model Editing · Depth: Expert, extended

Summary

TILDE (TILt-based Distributional Erasure) introduces a novel concept unlearning method for text-to-image diffusion models, addressing the critical need to suppress unwanted concepts while preserving image quality and diversity on benign generations. Unlike existing methods that often compromise retention for effective forgetting, TILDE formulates unlearning as a distributional alignment problem. It defines the desired post-unlearning state as a minimum-deviation conditional distribution from the pretrained model, achieved via an energy-tilted, anchor-free target. This principle is instantiated using residual \nabla-GFlowNet training, which learns the score correction induced by a thresholded CLIP-based forget energy. Evaluated on Stable Diffusion v1.5 across objects, artistic styles, and characters, TILDE demonstrates strong forgetting alongside improved retention and distributional fidelity compared to prior baselines.

Key takeaway

For AI Scientists and Machine Learning Engineers tasked with implementing concept unlearning in text-to-image diffusion models for compliance or safety, you should consider TILDE's principled approach. Its distributional alignment framework and residual \nabla-GFlowNet training offer a robust solution to achieve strong forgetting without sacrificing the quality and diversity of benign generations. This method helps you avoid the common trade-off between erasure efficacy and collateral damage, ensuring your models remain high-fidelity and compliant.

Key insights

Concept unlearning in diffusion models is best achieved by explicitly defining the desired post-unlearning distribution, not just suppressing concepts.

Principles

• Effective unlearning requires both strong forgetting and robust preservation of benign generation quality.
• The ideal unlearned model approximates a retain-only model trained without the forgotten data.
• Minimal-deviation principle: choose the distribution closest to the original that satisfies forgetting constraints.

Method

TILDE uses residual \nabla-GFlowNet training to learn score corrections from a thresholded CLIP-based forget energy, tilting the pretrained distribution to suppress concepts.

In practice

• Employ a thresholded CLIP score to define a no-penalty region for benign images.
• Integrate retain prompts during training as a regularizer for improved retention.
• Utilize LoRA adapters for efficient, parameter-space minimal updates.

Topics

• Text-to-Image Diffusion Models
• Concept Unlearning
• GFlowNets
• Machine Unlearning
• Generative AI Safety
• Distributional Alignment

Best for: Research Scientist, Computer Vision Engineer, AI Scientist, Machine Learning Engineer

Related on AIssential

• TILDE: TILt-based Distributional Erasure for Concept Unlearning — TILDE redefines concept unlearning as distributional alignment, improving forgetting and retention in diffusion models.
• Learning What to Forget: Improving LLM Unlearning via Learned Token-Level Importance — Effective LLM unlearning requires implicitly identifying and selectively targeting forget-specific tokens based on their conflict with retaining general capabilities.
• PURGE: Projected Unlearning via Retain-Guided Erasure — PURGE unlearns data by projecting gradients to preserve retain-set performance and erasing hidden representations towards retain distribution.
• Forgetting That Sticks: Quantization-Permanent Unlearning via Circuit Attribution — Quantization can reverse machine unlearning, necessitating methods that ensure forgetting persists post-compression.
• U-CAN: Utility-Aware Contrastive Attenuation for Efficient Unlearning in Generative Recommendation — U-CAN precisely unlearns sensitive data in GenRec LLMs by selectively attenuating LoRA adapter parameters based on activation contrasts and utility.
• POPS: Recovering Unlearned Multi-Modality Knowledge in MLLMs with Prompt-Optimized Parameter Shaking — POPS demonstrates that current Multi-modality Machine Unlearning methods are fundamentally vulnerable to adversarial recovery of sensitive data.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CV updates on arXiv.org.