Class Unlearning via Depth-Aware Removal of Forget-Specific Directions

2026-04-16 · Source: Computer Vision and Pattern Recognition · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Computer Vision & Pattern Recognition · Depth: Expert, quick

Summary

Machine unlearning aims to remove specific knowledge from trained models without full retraining. Existing class unlearning methods often fail to truly remove information, instead suppressing classifier heads or exhibiting weak selectivity, leaving forget-class structure in deep representations. A new method, DAMP (Depth-Aware Modulation by Projection), is introduced as a one-shot, closed-form weight-surgery technique. DAMP removes forget-specific directions from a pretrained network without gradient optimization. It computes class prototypes, extracts forget directions as residuals, and applies a projection-based update to reduce sensitivity. To maintain utility, DAMP uses a depth-aware scaling rule, applying smaller edits in early layers and larger edits in deeper layers. DAMP extends to multi-class forgetting via low-rank subspace removal and performs well across MNIST, CIFAR-10, CIFAR-100, and Tiny ImageNet, and on both convolutional and transformer architectures.

Key takeaway

For Computer Vision Engineers and Research Scientists developing or deploying models requiring selective knowledge removal, DAMP offers a robust and efficient solution. Its one-shot, closed-form approach avoids costly retraining and gradient-based optimization, making it ideal for scenarios where rapid, verifiable unlearning of specific classes is critical. You should consider DAMP to improve selective forgetting while preserving retain-class performance in your convolutional and transformer architectures.

Key insights

DAMP offers a one-shot, closed-form weight-surgery for class unlearning by removing forget-specific directions.

Principles

• True forgetting requires representational removal, not just classifier suppression.
• Depth-aware scaling preserves utility during unlearning edits.

Method

DAMP computes class prototypes, extracts forget directions as residuals, and applies a projection-based update with depth-aware scaling to remove forget-specific directions from network weights.

In practice

• Apply DAMP for efficient class unlearning in image classification.
• Use DAMP for multi-class forgetting via low-rank subspace removal.

Topics

• Machine Unlearning
• Class Unlearning
• DAMP
• Weight Surgery
• Deep Learning

Best for: Computer Vision Engineer, Research Scientist, AI Scientist, Machine Learning Engineer

Related on AIssential

• PURGE: Projected Unlearning via Retain-Guided Erasure — PURGE unlearns data by projecting gradients to preserve retain-set performance and erasing hidden representations towards retain distribution.
• Learning What to Forget: Improving LLM Unlearning via Learned Token-Level Importance — Effective LLM unlearning requires implicitly identifying and selectively targeting forget-specific tokens based on their conflict with retaining general capabilities.
• To forget is to preserve: Machine Unlearning for 3D medical image segmentation — Effective machine unlearning balances data erasure with preserving model utility for retained information.
• SHRED: Retain-Set-Free Unlearning via Self-Distillation with Logit Demotion — LLM unlearning can be retain-set-free by selectively demoting high-information tokens via self-distillation, preserving general utility.
• Forgetting That Sticks: Quantization-Permanent Unlearning via Circuit Attribution — Quantization can reverse machine unlearning, necessitating methods that ensure forgetting persists post-compression.
• CURaTE: Continual Unlearning in Real Time with Ensured Preservation of LLM Knowledge — CURaTE enables real-time LLM unlearning and knowledge preservation without modifying model parameters.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computer Vision and Pattern Recognition.