U-CAN: Utility-Aware Contrastive Attenuation for Efficient Unlearning in Generative Recommendation

2026-03-02 · Source: cs.LG updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Cybersecurity & Data Privacy · Depth: Expert, extended

Summary

U-CAN (Utility-aware Contrastive AttenuatioN) is a novel machine unlearning framework designed for Generative Recommendation (GenRec) systems that utilize Large Language Models (LLMs) with low-rank adapters (LoRA). GenRec models, when fine-tuned on user data, can inadvertently embed sensitive information, posing privacy risks. Existing unlearning methods often cause significant utility loss due to the "Polysemy Dilemma," where neurons handle both sensitive and general reasoning. U-CAN addresses this by quantifying risk through contrasting activations, focusing on neurons highly sensitive to the forgetting set but suppressed on the retention set. It incorporates a utility-aware calibration mechanism, combining weight magnitudes with retention-set activation norms to protect performance. Instead of binary pruning, U-CAN employs adaptive soft attenuation with a differentiable decay function to selectively down-scale high-risk parameters on LoRA adapters, preserving network connectivity. Experiments on ML-100k and Pantry datasets across seven metrics demonstrate U-CAN's strong privacy forgetting, utility retention, and computational efficiency.

Key takeaway

For Research Scientists developing or deploying LLM-based Generative Recommendation systems, U-CAN offers a robust solution to privacy concerns without sacrificing model utility. You should consider integrating U-CAN's approach to efficiently remove sensitive user data by leveraging its precision unlearning on LoRA adapters, which outperforms traditional gradient or pruning methods in balancing forgetting effectiveness with recommendation quality and operational efficiency. This framework allows for prompt responses to deletion requests while maintaining core recommendation capabilities.

Key insights

U-CAN precisely unlearns sensitive data in GenRec LLMs by selectively attenuating LoRA adapter parameters based on activation contrasts and utility.

Principles

• Targeted unlearning requires localizing risk at fine-grained activation levels.
• Preserving utility demands protecting parameters critical for general reasoning.
• Soft attenuation maintains network connectivity better than hard pruning.

Method

U-CAN uses contrastive activation analysis to identify sensitive neurons, a utility-aware calibration to score parameter importance, and adaptive soft attenuation with a continuous decay function to suppress high-risk LoRA parameters.

In practice

• Apply U-CAN to LoRA adapters for efficient unlearning in GenRec.
• Use contrastive activation to pinpoint privacy-sensitive neurons.
• Employ soft attenuation to avoid structural damage from hard pruning.

Topics

• Generative Recommendation
• Machine Unlearning
• Large Language Models
• LoRA Adapters
• Privacy Preservation

Best for: Research Scientist, AI Researcher, AI Scientist, Machine Learning Engineer

Related on AIssential

• TRACER: Token ReAssignment for Concept ERasure in Generative Recommendation — Generative recommendation unlearning requires token reassignment to resolve conflicts from shared semantic IDs, preserving utility.
• CURaTE: Continual Unlearning in Real Time with Ensured Preservation of LLM Knowledge — CURaTE enables real-time LLM unlearning and knowledge preservation without modifying model parameters.
• Benchmarking Empirical Privacy Protection for Adaptations of Large Language Models — Distribution shifts critically impact practical privacy in differentially private LLM adaptations, often undermining theoretical guarantees.
• Detoxifying LLMs via Representation Erasure-Based Preference Optimization — REPO robustly detoxifies LLMs by erasing toxic representations at a token level, resisting adaptive attacks.
• PURGE: Projected Unlearning via Retain-Guided Erasure — PURGE unlearns data by projecting gradients to preserve retain-set performance and erasing hidden representations towards retain distribution.
• Learning What to Forget: Improving LLM Unlearning via Learned Token-Level Importance — Effective LLM unlearning requires implicitly identifying and selectively targeting forget-specific tokens based on their conflict with retaining general capabilities.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.LG updates on arXiv.org.