Be Consistent! Enhancing Robust Visual Reasoning in LVLMs with Consistency Constraints

2026-07-27 · Source: cs.CV updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Multimodal AI · Depth: Advanced, extended

Summary

Researchers introduced ConVBench, a new benchmark designed to evaluate Large Vision-Language Models' (LVLMs) complex visual reasoning and logical consistency. ConVBench features images paired with two logically equivalent questions across six categories: action and state, complex counting, spatial reasoning, causal and intent understanding, commonsense reasoning, and temporal perception. To complement this, they developed ConVLM, an LVLM enhanced through Group Relative Policy Optimization (GRPO) with a novel consistency reward. ConVLM leverages automatically generated logically equivalent question-answer pairs and a dual reward system balancing accuracy and consistency. The 7B variant of ConVLM achieved 73.36% average Consistency and 66.83% Accuracy on ConVBench, significantly surpassing closed-source models like Claude-3.5-Sonnet-V2 by 11.12% and 20.94% respectively. ConVLM also demonstrated strong generalization, achieving 84.90% accuracy on V*Bench.

Key takeaway

For Machine Learning Engineers developing robust LVLMs, you should integrate consistency-based reinforcement learning into your training pipelines. This approach, using automatically generated logically equivalent question pairs and a dual accuracy-consistency reward, significantly improves model reliability and generalization beyond traditional supervised fine-tuning. Consider adopting GRPO with sample-wise consistency rewards to enhance both correctness and logical stability in complex visual reasoning tasks.

Key insights

LVLMs can achieve robust, consistent visual reasoning by training with logically equivalent question pairs and a dual accuracy-consistency reward.

Principles

• Logical consistency is a key robustness dimension.
• Consistency rewards improve both consistency and accuracy.
• GRPO-based training enhances generalization.

Method

ConVLM uses a two-stage process: GPT-4.1 generates logically equivalent question-answer pairs, then GRPO-based reinforcement learning optimizes LVLMs with a dual reward combining accuracy and consistency signals, even with noisy pseudo-labels.

In practice

• Generate logically equivalent QA pairs for training.
• Implement dual rewards for accuracy and consistency.
• Use GRPO for robust visual reasoning tasks.

Topics

• Large Vision-Language Models
• Visual Reasoning
• ConVBench
• Reinforcement Learning
• Consistency Constraints
• GRPO

Code references

• LiqiangJing/ConVLM

Best for: Research Scientist, Computer Vision Engineer, AI Scientist, Machine Learning Engineer

Related on AIssential

• Reasoning as Intersection: Consensus-Frame Alignment for Visual Focus in Video-MLLMs — Consensus Frame GRPO (CF-GRPO) improves Video-MLLM reasoning by aligning visual evidence with a consensus frame prior, using process-level rewards without temporal annotations.
• The Art of Interrogation: Consistency Amplifies Factuality in Spatial Reasoning — Spatial reasoning in LRMs can be amplified by self-supervised consistency training without ground-truth labels.
• Be Consistent! Enhancing Robust Visual Reasoning in LVLMs with Consistency Constraints — ConVBench and ConVLM improve LVLM visual reasoning robustness and logical consistency through a new benchmark and a GRPO-based reinforcement learning method.
• GRPO++: Tricks for Making RL Actually Work — Optimizing GRPO for LLM reasoning requires addressing inherent biases and system-level mismatches to achieve stable and efficient training.
• Cognitive Dual-Process Planning for Autonomous Driving with Structured Scene Knowledge and Verifiable Reasoning-Action Consistency — Adaptive dual-process VLM planning with structured knowledge and verifiable consistency improves autonomous driving performance and efficiency.
• SD-MAR: Multi-image Analytical Reasoning via Synthetic Data and Reinforcement Learning — Synthetic data and reinforcement learning significantly enhance Vision Language Models' multi-image analytical reasoning capabilities.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CV updates on arXiv.org.