PURGE: Projected Unlearning via Retain-Guided Erasure

2026-06-02 · Source: Artificial Intelligence · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Cybersecurity & Data Privacy · Depth: Expert, quick

Summary

PURGE is a novel machine unlearning algorithm introduced on 2026-06-02, which leverages the duality between continual learning and machine unlearning. It adapts gradient projection from A-GEM to ensure that each unlearning step does not increase the retain-set loss. The algorithm also incorporates multi-layer representation erasure, pushing forget-set activations in intermediate layers towards the retain distribution to remove information more deeply. A key design choice is its retain-confusion target, which aims for the model's natural confusion pattern on retain data instead of a uniform distribution, making unlearned models harder to distinguish from those retrained from scratch. PURGE employs two self-regulating stopping criteria—a retain-loss budget and a forget-accuracy target—eliminating the need for manual epoch tuning. Across experiments on five datasets (CIFAR-10, MNIST, SVHN, STL10, PathMNIST) and 22 class-level forgetting tasks, PURGE consistently maintained retain accuracy above 96% and achieved a Membership Inference Attack (MIA) AUROC close to 0.5, outperforming several published baselines.

Key takeaway

For Machine Learning Engineers developing models requiring data erasure, PURGE offers a robust unlearning method that significantly enhances privacy without sacrificing utility. You should consider integrating its gradient projection and retain-confusion targeting approach to achieve MIA AUROC close to 0.5 while maintaining over 96% retain accuracy. This approach also simplifies deployment by eliminating manual epoch tuning, streamlining your unlearning pipeline.

Key insights

PURGE unlearns data by projecting gradients to preserve retain-set performance and erasing hidden representations towards retain distribution.

Principles

• Machine unlearning and continual learning are dual problems.
• Target retain-confusion, not uniform distribution, for privacy.
• Self-regulating criteria remove manual tuning.

Method

PURGE adapts A-GEM's gradient projection to constrain unlearning steps, preventing retain-set loss increase. It performs multi-layer representation erasure, pushing forget-set activations towards the retain distribution.

In practice

• Implement gradient projection for unlearning.
• Use retain-confusion as an unlearning target.
• Apply multi-layer representation erasure.

Topics

• Machine Unlearning
• Continual Learning
• Gradient Projection
• Representation Erasure
• Data Privacy
• Membership Inference Attacks

Best for: Research Scientist, CTO, VP of Engineering/Data, AI Scientist, Machine Learning Engineer, AI Security Engineer

Related on AIssential

• Forgetting That Sticks: Quantization-Permanent Unlearning via Circuit Attribution — Quantization can reverse machine unlearning, necessitating methods that ensure forgetting persists post-compression.
• Interference-Aware Multi-Task Unlearning — Multi-task unlearning requires mitigating task- and instance-level interference to preserve retained performance and generalization.
• CURaTE: Continual Unlearning in Real Time with Ensured Preservation of LLM Knowledge — CURaTE enables real-time LLM unlearning and knowledge preservation without modifying model parameters.
• Class Unlearning via Depth-Aware Removal of Forget-Specific Directions — DAMP offers a one-shot, closed-form weight-surgery for class unlearning by removing forget-specific directions.
• Graph Propagated Projection Unlearning: A Unified Framework for Vision and Audio Discriminative Models — GPPU efficiently unlearns specific classes from deep networks by geometrically projecting features and fine-tuning.
• U-CAN: Utility-Aware Contrastive Attenuation for Efficient Unlearning in Generative Recommendation — U-CAN precisely unlearns sensitive data in GenRec LLMs by selectively attenuating LoRA adapter parameters based on activation contrasts and utility.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Artificial Intelligence.