LASER: A Corrective Lens for LVLMs via Visual Attention Preservation and Sink Suppression

2026-07-02 · Source: Computer Vision and Pattern Recognition · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Computer Vision · Depth: Expert, quick

Summary

LASER is a novel post-training framework designed to mitigate "visual forgetting" in Large Vision-Language Models (LVLMs) during long-horizon decoding. This issue, where attention drifts from visual evidence, has been re-evaluated, revealing that performance degradation stems from early-stage attention decay and concentration on task-irrelevant visual sink tokens, rather than solely late-stage decay. LASER addresses these factors by regulating both the visual attention trajectory and intra-visual token attention distribution. It employs two complementary rewards: a Visual Grounding Reward, which ensures sustained attention on semantically salient visual tokens, and a Sink Suppression Reward, which prevents excessive focus on uninformative sink tokens. This dual approach preserves early-stage grounding and prevents attention collapse. Extensive experiments across eight benchmark datasets demonstrate LASER's consistent superior performance over existing baselines, validating attention-aware training as an effective solution.

Key takeaway

For Machine Learning Engineers developing Large Vision-Language Models, if you are struggling with visual forgetting during long-horizon decoding, consider implementing attention-aware post-training frameworks like LASER. Your models' performance can significantly improve by explicitly regulating visual attention trajectory and suppressing focus on uninformative sink tokens. This approach directly addresses early-stage attention decay, ensuring more robust visual grounding and preventing attention collapse in complex reasoning tasks.

Key insights

LVLM visual forgetting stems from early attention decay and sink token concentration, correctable via attention-aware training.

Principles

• Early attention decay disrupts visual evidence acquisition.
• Task-irrelevant sink tokens concentrate visual attention.
• Regulate attention trajectory for robust LVLM reasoning.

Method

LASER applies a post-training framework using a Visual Grounding Reward to sustain attention on salient visual tokens and a Sink Suppression Reward to penalize focus on uninformative sink tokens.

In practice

• Integrate attention-aware training into LVLM pipelines.
• Analyze early-stage visual attention patterns.
• Develop mechanisms to suppress visual sink token focus.

Topics

• Large Vision-Language Models
• Visual Forgetting
• Attention Mechanisms
• Post-training Frameworks
• Visual Grounding
• Sink Suppression

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, Computer Vision Engineer

Related on AIssential

• Visual Latents Know More Than They Say: Unsilencing Latent Reasoning in MLLMs — Visual latents in MLLMs can be "silenced" by training objectives, but their reasoning capacity can be unleashed at inference time.
• Persistent Visual Memory: Sustaining Perception for Deep Generation in LVLMs — Persistent Visual Memory (PVM) counteracts visual signal dilution in LVLMs by providing sustained, on-demand visual perception.
• AIM: Asymmetric Information Masking for Visual Question Answering Continual Learning — Asymmetric Information Masking (AIM) prevents catastrophic forgetting in VQA by balancing VLM stability and plasticity.
• StrLoRA: Towards Streaming Continual Visual Instruction Tuning for MLLMs — Textual instructions provide a stable signal for expert routing in streaming multimodal learning.
• Mitigating Visual Hallucinations in Multimodal Systems through Retrieval-Augmented Reliability-Aware Inference — Retrieval-augmented reliability-aware inference significantly reduces visual hallucinations in MLLMs without retraining.
• MLLMs Get It Right, Then Get It Wrong: Tracing and Correcting Late-Layer Textual Bias — MLLMs often correctly process visual information in early layers, but a "late-layer textual override" bias shifts final predictions towards text.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computer Vision and Pattern Recognition.