Interference-Aware Multi-Task Unlearning

2026-05-21 · Source: cs.AI updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Cybersecurity & Data Privacy, Robotics & Autonomous Systems · Depth: Expert, extended

Summary

The paper introduces multi-task unlearning, addressing challenges in removing specific data contributions from models operating across multiple tasks with shared backbones. It defines two settings: full-task unlearning, removing a target instance from all tasks, and partial-task unlearning, removing supervision only from selected tasks. The authors identify task-level and instance-level interference caused by shared parameters coupling forget and retain sets. To mitigate this, they propose an interference-aware framework combining task-aware gradient projection, which constrains updates to task-specific subspaces, with instance-level gradient orthogonalization, reducing conflicts between forget and retain signals. Experiments on NYUv2 and PASCAL multi-task computer vision benchmarks across five tasks demonstrate the method's effectiveness, reducing Unlearning Impact Score (UIS) by 30.3% in full-task unlearning and 52.9% in partial-task unlearning compared to the strongest baseline, while maintaining strong generalization and scalability up to 50% unlearn ratios.

Key takeaway

For machine learning engineers implementing unlearning in multi-task models, directly applying single-task methods will degrade performance significantly. You should adopt interference-aware strategies like gradient projection and orthogonalization to selectively remove data influence without harming other tasks or retained data. This approach ensures compliance with privacy regulations like GDPR while maintaining model utility and generalization, even with large unlearn ratios.

Key insights

Multi-task unlearning requires mitigating task- and instance-level interference to preserve retained performance and generalization.

Principles

• Shared parameters couple forget and retain sets.
• Unlearning induces task- and instance-level interference.
• Direct gradient updates are often suboptimal.

Method

The framework combines task-aware gradient projection to constrain updates to task-specific subspaces and instance-level gradient orthogonalization to remove conflicting components between forget and retain gradients.

In practice

• Use low-rank updates for parameter efficiency.
• Regularize task-specific subspaces for orthogonality.
• Sequentially orthogonalize against retain gradients.

Topics

• Machine Unlearning
• Multi-Task Learning
• Gradient Projection
• Parameter-Efficient Fine-Tuning
• Computer Vision
• Data Privacy

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, AI Security Engineer

Related on AIssential

• PURGE: Projected Unlearning via Retain-Guided Erasure — PURGE unlearns data by projecting gradients to preserve retain-set performance and erasing hidden representations towards retain distribution.
• Fair Cognitive Impairment Detection Through Unlearning — Demographic unlearning via gradient reversal in multimodal fusion improves fair and robust MCI detection from speech.
• CATA: Continual Machine Unlearning via Conflict-Averse Task Arithmetic — CATA enables continual machine unlearning for VLMs by aggregating unlearning task vectors to prevent knowledge re-emergence.
• Essential Subspace Merging for Multi-Task Learning — Model merging interference is reduced by isolating and fusing high-energy "essential subspaces" of task updates.
• Graph Propagated Projection Unlearning: A Unified Framework for Vision and Audio Discriminative Models — GPPU efficiently unlearns specific classes from deep networks by geometrically projecting features and fine-tuning.
• Auditing Machine Unlearning: A Systematic Research on Whether Models Truly Forget — A new auditing framework reliably verifies machine unlearning effectiveness, revealing varying success across different unlearning methods.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.AI updates on arXiv.org.