ReFoCUS: Reinforcement-guided Frame Optimization for Contextual Understanding

2026-06-12 · Source: cs.AI updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Computer Vision · Depth: Expert, extended

Summary

ReFoCUS (Reinforcement-guided Frame Optimization for Contextual UnderStanding) is a new reinforcement learning framework designed to enhance video-LLM performance by optimizing visual input selection. Unlike traditional methods that fine-tune textual outputs, ReFoCUS learns a frame selection policy using reward signals derived from a reference LMM, such as InternVL3, to identify frames intrinsically preferred by the model for accurate, temporally grounded responses. This approach avoids explicit frame-level supervision and employs an autoregressive, conditional selection architecture to navigate the vast combinatorial frame space efficiently. The framework consistently improves reasoning performance across benchmarks like Video-MME, LongVideoBench, MLVU, and Video-MMMU, demonstrating significant gains in long video understanding and knowledge acquisition tasks. It effectively pinpoints semantically relevant frames, even in "needle-in-a-haystack" scenarios, and adapts its selection strategy based on query semantics.

Key takeaway

For Machine Learning Engineers developing video-LLMs, your strategy should extend beyond textual output optimization to include visual input selection. Implementing reinforcement learning for frame optimization, as demonstrated by ReFoCUS, can significantly enhance contextual understanding and reasoning, particularly for long-form video content. This approach allows models to internalize visual preferences, leading to more accurate predictions and improved performance across diverse video QA benchmarks, all without costly frame-level supervision.

Key insights

Optimizing visual input selection via reinforcement learning significantly enhances video-LLM contextual understanding and reasoning.

Principles

• Align frame selection with model's intrinsic visual preferences.
• Reinforcement learning can optimize input-level policies.
• Autoregressive selection ensures temporal coherence.

Method

ReFoCUS trains a policy model to autoregressively select frame subsets, guided by margin-based rewards from a frozen reference LMM's prediction confidence. This process uses filtered QA pairs to ensure stable learning.

In practice

• Use LMM output logits as implicit frame feedback.
• Filter low-variance QA pairs for stable RL training.
• Employ autoregressive frame selection for efficiency.

Topics

• Video-LLMs
• Reinforcement Learning
• Frame Selection
• Policy Optimization
• Contextual Understanding
• Multi-modal AI

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, Computer Vision Engineer

Related on AIssential

• VideoSEG-O3: A Multi-turn Reinforcement Learning Framework for Reasoning Video Object Segmentation — Multi-turn RL with calibrated segmentation feedback enables adaptive, precise video object segmentation.
• EvoVid: Temporal-Centric Self-Evolution for Video Large Language Models — EvoVid enables Video-LLMs to self-evolve temporal reasoning from raw video, eliminating human annotation dependency.
• Reinforced Fast Weights with Next-Sequence Prediction — REFINE improves fast weight models for long-context tasks by shifting from next-token to next-sequence prediction via reinforcement learning.
• Learning to Select Visual In-Context Demonstrations — Learned, diversity-aware demonstration selection significantly improves MLLM performance on objective visual regression tasks.
• GUIDE: Guided Updates for In-context Decision Evolution in LLM-Driven Spacecraft Operations — GUIDE enables LLMs to adapt in real-time by evolving natural-language decision rules without weight updates.
• Performance-Driven Policy Optimization for Speculative Decoding with Adaptive Windowing — Window-level optimization via reinforcement learning significantly improves speculative decoding efficiency for LLMs.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.AI updates on arXiv.org.