Learning Spatiotemporal Sensitivity in Video LLMs via Counterfactual Reinforcement Learning

2026-05-22 · Source: cs.CV updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Video Understanding · Depth: Expert, extended

Summary

Video large language models (Video LLMs) often rely on static shortcuts, like single-frame cues or language priors, instead of genuine spatiotemporal understanding. To address this, Counterfactual Relational Policy Optimization (CRPO) is proposed, a dual-branch reinforcement learning framework. CRPO constructs counterfactual videos using horizontal flips and temporal reversals, training on both original and transformed branches. It introduces a Counterfactual Relation Reward (CRR) that encourages answers to change for dynamic questions and remain unchanged for static questions, making shortcut policies difficult to consistently reward. To evaluate this, DyBench, a paired counterfactual video benchmark with 3,014 videos, was created, covering reversible dynamics, moving direction, and event sequence, using a strict pair-accuracy metric. Experiments on Qwen3-VL-8B show CRPO improves DyBench P-Acc by +7.7 and TimeBlind I-Acc by +8.2 over the base model, demonstrating enhanced spatiotemporal sensitivity.

Key takeaway

For machine learning engineers developing Video LLMs, you should integrate counterfactual reinforcement learning to overcome static shortcut reliance. CRPO's dual-branch approach, which rewards expected answer changes or invariances across original and transformed videos, directly enhances spatiotemporal sensitivity. This method, validated by DyBench, ensures your models genuinely track video dynamics, leading to more robust and reliable performance in applications requiring true temporal understanding.

Key insights

Video LLMs need counterfactual training to develop genuine spatiotemporal sensitivity beyond static shortcuts.

Principles

• Spatiotemporal sensitivity requires equivariance for dynamic questions and invariance for static ones.
• Correctness-only rewards in RL can reinforce shortcut policies in Video LLMs.

Method

CRPO uses a dual-branch GRPO framework with a Task Router to select transformations (horizontal flip, temporal reversal) for counterfactual videos. It applies a Counterfactual Relation Reward (CRR) to both original and counterfactual branches, rewarding expected answer relations.

In practice

• Construct counterfactual video pairs by horizontal flip or temporal reversal.
• Evaluate spatiotemporal understanding using paired accuracy metrics like DyBench P-Acc.

Topics

• Video LLMs
• Reinforcement Learning
• Counterfactual Learning
• Spatiotemporal Understanding
• DyBench Benchmark
• Qwen3-VL

Code references

• open-compass/VLMEvalKit

Best for: Research Scientist, AI Scientist, Machine Learning Engineer

Related on AIssential

• Reducing Credit Assignment Variance via Counterfactual Reasoning Paths — Counterfactual trajectory comparison enables process-level credit assignment, reducing gradient variance in LLM reinforcement learning.
• TeDiO: Temporal Diagonal Optimization for Training-Free Coherent Video Diffusion — Temporal coherence in video diffusion models correlates with smooth, band-diagonal self-attention patterns.
• DynaCF: Mitigating Shortcut Learning in Reward Models via Dynamic Counterfactual Sensitivity — DynaCF dynamically reweights reward model training samples to reduce reliance on superficial shortcuts.
• Reasoning as Intersection: Consensus-Frame Alignment for Visual Focus in Video-MLLMs — Consensus Frame GRPO (CF-GRPO) improves Video-MLLM reasoning by aligning visual evidence with a consensus frame prior, using process-level rewards without temporal annotations.
• GEOALIGN: Geometric Rollout Curation for Robust LLM Reinforcement Learning — GEOALIGN stabilizes LLM RL by curating rollouts to resolve directional inconsistencies from conflicting reward signals.
• Counterfactual Conditional Likelihood Rewards for Multiagent Exploration — CCL rewards improve multiagent exploration by quantifying each agent's unique contribution to joint team observation.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CV updates on arXiv.org.