ICED: Concept-level Machine Unlearning via Interpretable Concept Decomposition

2026-05-15 · Source: cs.CV updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Cybersecurity & Data Privacy · Depth: Expert, extended

Summary

ICED (Interpretable ConcEpt Decomposition) is a novel machine unlearning framework for Vision-Language Models (VLMs) like CLIP, addressing the limitations of instance-level unlearning which often removes unrelated semantics. ICED constructs a compact, task-specific concept vocabulary from the forgetting set using a multimodal large language model (MLLM). It then decomposes visual representations into sparse, non-negative combinations of these semantic concepts, enabling fine-grained knowledge manipulation. The method formulates unlearning as a concept-level optimization problem, selectively suppressing target concepts while preserving intra-instance non-target semantics and global cross-modal knowledge. Experiments on CIFAR-10 and ImageNet-1K, using CLIP RN50 and RN101 backbones, demonstrate that ICED achieves more comprehensive target forgetting and better preserves non-target knowledge and model utility compared to existing VLM unlearning methods, with a superior average score across various benchmarks.

Key takeaway

For research scientists and engineers developing or deploying Vision-Language Models, ICED offers a more precise approach to machine unlearning. If you need to remove specific concepts (e.g., sensitive data, copyrighted content) from a VLM without degrading its general utility or unrelated knowledge, consider implementing concept-level decomposition and optimization. This method significantly reduces collateral damage compared to traditional instance-level unlearning, making your models more compliant and robust to data removal requests.

Key insights

Concept-level unlearning in VLMs precisely removes target knowledge by decomposing visual representations into semantic concepts.

Principles

• Instance-level unlearning is too coarse for VLMs.
• Decompose visual representations into sparse concept bases.
• Balance forgetting, intra-instance, and global preservation.

Method

ICED constructs a task-specific concept vocabulary via MLLM, aligns modalities, and decomposes visual features into sparse concept combinations. It then optimizes three loss functions for forgetting, intra-instance preservation, and global knowledge retention.

In practice

• Use MLLMs to build task-specific concept vocabularies.
• Apply $\ell_1$ sparsity regularization for concept decomposition.
• Combine forgetting, intra-instance, and global preservation losses.

Topics

• Machine Unlearning
• Vision-Language Models
• Interpretable Concept Decomposition
• CLIP Models
• Multimodal Large Language Models

Code references

• openai/CLIP

Best for: Computer Vision Engineer, Research Scientist, AI Scientist, Machine Learning Engineer, MLOps Engineer

Related on AIssential

• On the Limits of Token Reduction for Efficient Unified Vision Language Training — Unified VLM training efficiency is limited by task-specific token reduction due to synergy loss from divergent parameter pathways.
• Livecoding: ICE and the Factored Cognition Primer by Ought — Iterated decomposition and execution tracing significantly enhance language model reliability and debugging.
• Beyond Scalar Distances: Semantic Attribute Gradients from Frozen MLLMs for Visual Embeddings — SAGA uses frozen MLLMs to provide attribute-resolved supervision for vision encoders, moving beyond scalar distances.
• RepSelect: Robust LLM Unlearning via Representation Selectivity — RepSelect achieves robust LLM unlearning by selectively targeting forget-set-specific representations, preventing easy reversal.
• Cross-Modal Masked Compositional Concept Modeling for Enhancing Visio-Linguistic Compositionality — Masking and reconstructing cross-modal compositional concepts improves VLM understanding of relations, attributes, and word order.
• Dynamic Large Concept Models: Latent Reasoning in an Adaptive Semantic Space — DLCMs improve LLM efficiency by shifting computation from tokens to a compressed, concept-based reasoning space.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CV updates on arXiv.org.