Modality-Decoupled Online Recursive Editing

2026-05-22 · Source: cs.LG updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, extended

Summary

Modality-Decoupled Online Recursive Editing (M-ORE) is introduced as a novel framework for lifelong adaptation in Multimodal Large Language Models (MLLMs), specifically designed to overcome challenges in online model editing. It addresses "cross-modal conflict," where visually dominant activations skew update statistics, and "inter-edit interference," where sequential writes become entangled. M-ORE, derived from a unified proximal-projection formulation, employs a closed-form update with Sherman-Morrison recursion, ensuring constant per-edit overhead. It maintains module-wise locality statistics for text and visual components and performs continual updates in a fixed orthogonal low-rank edit subspace. Experiments on BLIP2-OPT (2.7B) and LLaVA-v1.5 (7B) across E-VQA and E-IC benchmarks demonstrate M-ORE's consistent improvements in reliability, generality, and locality over strong baselines, particularly under long edit horizons (e.g., 100 edits), while preserving general capabilities and achieving favorable quality-efficiency scaling.

Key takeaway

For MLOps Engineers managing evolving knowledge in deployed Multimodal Large Language Models, you should consider M-ORE for its ability to provide constant-overhead online updates. This method effectively mitigates cross-modal conflict and inter-edit interference, ensuring reliability and generality over long edit horizons. Implementing M-ORE can significantly reduce retraining costs and maintain model performance without accumulating drift or increasing memory footprint.

Key insights

M-ORE resolves MLLM online editing issues by decoupling modalities and using a fixed orthogonal low-rank subspace for constant-overhead updates.

Principles

• Decouple modality-specific update statistics.
• Utilize a fixed orthogonal low-rank write space.
• Suppress updates on frequently activated coordinates.

Method

M-ORE employs a unified proximal-projection with a Sherman-Morrison recursion for closed-form updates. It maintains module-wise locality statistics and performs continual updates in a fixed orthogonal low-rank edit subspace.

In practice

• Maintain separate locality statistics for visual and text modules.
• Implement edits within a fixed orthogonal low-rank subspace.
• Use Sherman-Morrison recursion for efficient, drift-free updates.

Topics

• Multimodal Large Language Models
• Model Editing
• Online Learning
• Sherman-Morrison Recursion
• Low-Rank Adaptation
• Catastrophic Forgetting

Code references

• lab-klc/M-ORE
• zjunlp/EasyEdit

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, MLOps Engineer

Related on AIssential

• MODF-SIR: A Multi-agent Omni-modal Distilled Framework for Social Intelligence Reasoning — A multi-agent, omni-modal framework uses distillation and TTA with LoRA for superior social intelligence reasoning.
• Emerging Flexible Designs for Geospatial Multimodal Foundation Models — Apples-to-apples comparison of geospatial foundation model architectures reveals design trade-offs in flexibility, alignment, and performance.
• A Data-Efficient Path to Multilingual LLMs: Language Expansion via Post-training PARAM$Δ$ Integration into Upcycled MoE — Upcycling dense LLMs into MoE architectures with parameter delta integration efficiently expands language capabilities.
• A Mixture-of-Experts Model for Multimodal Emotion Recognition in Conversations — MiSTER-E uses a Mixture-of-Experts framework to fuse speech and text for robust emotion recognition in conversations.
• Same Evidence, Different Answer: Auditing Order Sensitivity in Multimodal Large Language Models — Multimodal LLMs exhibit significant order sensitivity, with current prompt-level mitigations proving insufficient for general robustness.
• MLUBench: A Benchmark for Lifelong Unlearning Evaluation in MLLMs — MLLM lifelong unlearning uniquely challenges multimodal alignment, requiring isolated, modular solutions to prevent cumulative degradation.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.LG updates on arXiv.org.