Towards Knowledge Alignment in Code LLMs: Contrastive Unlearning for Evolving APIs

2026-07-01 · Source: cs.SE updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering · Depth: Expert, extended

Summary

CURE, a novel contrastive unlearning approach, addresses the critical issue of Large Language Models (LLMs) generating deprecated API usages in code, which often results in unreliable and incompatible software. Traditional fine-tuning methods are inefficient for localized API modifications, and existing unlearning techniques primarily suppress outdated knowledge without explicitly guiding models toward correct replacements. CURE tackles this by jointly discouraging deprecated APIs and actively promoting their valid alternatives. Evaluated on four Code LLMs—DeepSeek-Coder-1.3B, StarCoder2-3B, CodeLlama-7B, and CodeGen-2B—using a recent deprecated API benchmark, CURE demonstrated superior performance. It significantly reduced deprecated API usage, substantially improved correct API replacement rates, and maintained general code generation capabilities on HumanEval, outperforming two state-of-the-art baselines. This highlights the importance of combining suppression with replacement-aware learning for adapting LLMs to evolving software ecosystems.

Key takeaway

For Machine Learning Engineers adapting Code LLMs to rapidly evolving software libraries, you should integrate contrastive unlearning methods like CURE. Relying solely on suppression-based unlearning risks generating mismatched or incomplete code. Instead, explicitly guide your models to both forget deprecated APIs and adopt correct replacements. This approach significantly improves code reliability and maintains general generation performance, ensuring your models remain current and functional.

Key insights

CURE uses contrastive unlearning to both suppress deprecated APIs and promote valid replacements in Code LLMs.

Principles

• Suppressing deprecated APIs alone is insufficient.
• Explicitly promoting replacements improves utility.
• Token-level suppression is more effective.

Method

CURE extends gradient-based parametric unlearning (SimNPO/PROD) with a contrastive objective. It constructs positive samples with valid APIs to enforce preference for correct alternatives over deprecated ones.

In practice

• Construct contrastive datasets with deprecated and valid API pairs.
• Apply token-level unlearning for precise API modification.
• Evaluate unlearning impact on general code generation.

Topics

• Code LLMs
• Machine Unlearning
• API Evolution
• Contrastive Learning
• Software Engineering
• Code Generation

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, AI Engineer

Related on AIssential

• RepSelect: Robust LLM Unlearning via Representation Selectivity — RepSelect achieves robust LLM unlearning by selectively targeting forget-set-specific representations, preventing easy reversal.
• Automated Root-Cause Subclassification and No-Code Fix Generation for Invalid Bug Reports — Automating invalid bug report subclassification and no-code fix generation significantly reduces software maintenance overhead.
• Reward-Free Code Alignment from Pretrained or Fine-Tuned LLM: Unpacking the Trade-offs for Code Generation — Reward-free alignment significantly improves code LLM non-functional properties, with pretrained models showing higher relative gains and finetuned models offering better absolute performance.
• Detoxification for LLM: From Dataset Itself — Detoxifying LLM pretraining datasets directly at the source fundamentally reduces inherent model toxicity while preserving semantics.
• Code-A1: Adversarial Evolving of Code LLM and Test LLM via Reinforcement Learning — Adversarial co-evolution of Code LLMs and Test LLMs improves code generation and test quality by separating objectives.
• MLUBench: A Benchmark for Lifelong Unlearning Evaluation in MLLMs — MLLM lifelong unlearning uniquely struggles with multimodal alignment degradation, which MLUBench and LUMoE address.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.SE updates on arXiv.org.