DualOptim+: Bridging Shared and Decoupled Optimizer States for Better Machine Unlearning in Large Language Models

2026-05-23 · Source: cs.LG updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Large Language Models · Depth: Expert, extended

Summary

DualOptim+ is a novel optimization framework designed to enhance machine unlearning in large language models (LLMs). It introduces a base state to capture common representations shared by forgetting and retaining objectives, alongside decoupled delta states that preserve objective-specific residuals. This architecture adaptively bridges shared and decoupled states based on directional gradient conflicts. The framework also offers DualOptim+ 8bit, a quantized variant that reduces memory overhead without performance compromise. Extensive experiments across fictitious and real-world unlearning, safety alignment, and multi-task learning tasks demonstrate DualOptim+'s consistent achievement of a superior trade-off between different objectives, outperforming baselines like Joint, Alternate, and DualOptim. For instance, DualOptim+ 8bit reduces memory to 33.68 GB/GPU from 39.04 GB/GPU for its 32-bit counterpart.

Key takeaway

For AI Scientists and ML Engineers tackling LLM unlearning or multi-objective optimization, DualOptim+ offers a robust solution to balance conflicting goals. You should implement DualOptim+ for a superior trade-off between knowledge erasure and model utility, especially with dynamic gradient conflicts. Its 8-bit quantized variant provides significant memory savings, making it practical for resource-constrained environments without sacrificing performance.

Key insights

DualOptim+ adaptively balances shared and decoupled optimizer states for superior LLM unlearning.

Principles

• Decompose optimizer states into shared base and objective-specific delta components.
• Adaptively bridge shared and decoupled states based on gradient conflict.
• Quantize optimizer states (8-bit) to reduce memory overhead without performance loss.

Method

DualOptim+ updates a base state with joint gradients and delta states with residuals (gradient minus base state). Parameters combine base and respective delta states. This adapts to gradient conflict.

In practice

• Apply to machine unlearning for specific data erasure.
• Use for safety alignment to remove harmful knowledge.
• Extend to multi-task learning for objective balancing.

Topics

• Machine Unlearning
• Large Language Models
• Optimizer States
• Gradient Conflict
• Quantization
• Safety Alignment
• Multi-task Learning

Code references

• CityU-MLO/DualOptimPlus
• tatsu-lab/stanford_alpaca
• meta-llama/PurpleLlama
• sail-sg/closer-look-LLM-unlearning
• EleutherAI/lm-evaluation-harness

Best for: AI Engineer, Research Scientist, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• From 32 Bits to 1.58: The Illustrated Guide to LLM Quantization — LLM quantization significantly reduces memory and accelerates inference while maintaining intelligence across various bit precisions.
• Correcting Stochastic Update Bias in Preconditioned Language Model Optimizers — Stochastic preconditioned optimizers suffer from two finite-sample biases that can be corrected for improved language model training.
• Spectra: Rethinking Optimizers for LLMs Under Spectral Anisotropy — LLM gradients exhibit a "spike-tail" anisotropy where dominant linguistic structures suppress long-tail semantic learning.
• A Dual-Path Architecture for Scaling Compute and Capacity in LLMs — A dual-path architecture scales LLM compute and capacity simultaneously using deep and wide sublayers with per-token gates.
• Zero-order Parameter-free Optimization for LMO-based Methods: Novel Approach for Efficient Fine-tuning — AdaNAGED unifies gradient-free, adaptive, and geometry-aware optimization for memory-efficient LLM fine-tuning.
• Scaling Behaviors of LLM Reinforcement Learning Post-Training: An Empirical Study in Mathematical Reasoning — Larger LLMs demonstrate superior efficiency and performance in RL post-training for mathematical reasoning.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.LG updates on arXiv.org.