Hidden Forgetting in Continual Multimodal Learning: When Accuracy Survives but Grounding Fails

2026-07-02 · Source: Artificial Intelligence · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, quick

Summary

A new study identifies "hidden evidence-use forgetting" in multimodal large language models (MLLMs) undergoing continual adaptation. While standard metrics focus on answer accuracy, this overlooked failure mode reveals MLLMs can retain correct answers yet silently shift their reliance on visual, textual, OCR, chart, and document evidence channels. To address this, researchers propose \textsc{RCL}, a replay-free reliance-constrained continual learning framework. \textsc{RCL} operates by freezing a previous checkpoint as a behavioral reference, estimating evidence-reliance profiles through counterfactual channel interventions, and jointly optimizing task learning, prediction preservation, and reliance preservation without increasing inference-time costs. Evaluated across CoIN, COAST, MCITlib, and an evidence-sensitive multimodal stream, \textsc{RCL} consistently improved final performance, reduced forgetting, and substantially lowered modality reliance drift, dominant evidence flips, and hidden forgetting rates. This work, published on 2026-07-02, emphasizes preserving the evidence path for robust continual multimodal learning.

Key takeaway

For Machine Learning Engineers developing continually adapting multimodal LLMs, you should prioritize evaluating and preserving the model's evidence-use stability, not just its answer accuracy. Hidden forgetting can silently degrade how your models ground their responses. Consider integrating reliance-constrained continual learning frameworks like \textsc{RCL} to maintain robust evidence paths, ensuring your MLLMs adapt effectively without compromising their interpretability or trustworthiness over time.

Key insights

Continual multimodal learning must preserve evidence-use stability, not just answer accuracy, to prevent hidden forgetting.

Principles

• MLLMs can exhibit "hidden evidence-use forgetting" where accuracy persists but evidence grounding shifts.
• Continual learning evaluation should include evidence-reliance stability.

Method

The \textsc{RCL} framework freezes a prior model as a reference, estimates evidence-reliance profiles using counterfactual channel interventions, and jointly optimizes task learning, prediction, and reliance preservation.

In practice

• Implement \textsc{RCL} to improve MLLM final performance and reduce forgetting.
• Use \textsc{RCL} to lower modality reliance drift and dominant evidence flips in MLLM adaptation.

Topics

• Continual Learning
• Multimodal LLMs
• Evidence Grounding
• Hidden Forgetting
• RCL Framework
• Counterfactual Interventions

Best for: Research Scientist, AI Scientist, Machine Learning Engineer

Related on AIssential

• Repurposing Adversarial Perturbations for Continual Learning: From Defense to Active Alignment — AdvCL repurposes adversarial perturbations as a geometric control signal to enhance continual learning stability and robustness.
• MAny: Merge Anything for Multimodal Continual Instruction Tuning — Catastrophic forgetting in MLLMs stems from dual perception and reasoning degradation, addressable via training-free merging.
• Lost or Hidden? A Concept-Level Forgetting in Supervised Continual Learning — Catastrophic forgetting in continual learning is often representational misalignment, not complete information erasure.
• Teaching AI to Remember: What I Learned Building a Continual Learning Framework for Drug Discovery — Continual learning models can avoid catastrophic forgetting by integrating new knowledge with awareness of existing representations.
• Learning, Fast and Slow: Towards LLMs That Adapt Continually [R] — Combining slow parameter updates with fast context optimization enhances LLM adaptation and mitigates forgetting.
• Staying VIGILant: Mitigating Visual Laziness via Counterfactual Visual Alignment in MLLMs — VIGIL mitigates MLLM visual laziness and hallucinations by causally grounding responses to visual input, penalizing "blind confidence" via counterfactual alignment.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Artificial Intelligence.