Be Consistent! Enhancing Robust Visual Reasoning in LVLMs with Consistency Constraints

2026-07-23 · Source: Computer Vision and Pattern Recognition · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Computer Vision & Pattern Recognition · Depth: Expert, quick

Summary

Large Vision-Language Models (LVLMs) struggle with robust visual reasoning, a gap existing benchmarks fail to fully address. To tackle this, a new benchmark called ConVBench is introduced, featuring images paired with two logically equivalent questions across six categories: action and state, complex counting, spatial reasoning, causal and intent understanding, commonsense reasoning, and temporal perception. ConVBench also defines new evaluation metrics: logical consistency and robust accuracy, which jointly assess correctness and consistency. Complementing this, ConVLM is presented, a method that enhances LVLM reasoning using Group Relative Policy Optimization (GRPO)-based reinforcement learning. ConVLM employs a novel consistency reward, leveraging automatically generated logically equivalent question-answer pairs and a dual-reward design combining accuracy- and consistency-based signals to promote agreement between paired responses, functioning effectively with or without strict answer supervision.

Key takeaway

For Machine Learning Engineers developing or evaluating Large Vision-Language Models, you should integrate ConVBench to rigorously assess complex visual reasoning and logical consistency, moving beyond simple vision-centric tasks. Consider implementing ConVLM's GRPO-based reinforcement learning with its dual accuracy and consistency rewards to significantly enhance your models' robustness and agreement across logically equivalent queries, even without strict answer supervision.

Key insights

ConVBench and ConVLM improve LVLM visual reasoning robustness and logical consistency through a new benchmark and a GRPO-based reinforcement learning method.

Principles

• Logical consistency is critical for reliable reasoning.
• Dual-reward designs enhance response agreement.
• Auto-generated Q&A pairs train consistency.

Method

ConVLM applies GRPO-based reinforcement learning with a novel consistency reward. It uses automatically generated logically equivalent Q&A pairs and a dual accuracy- and consistency-based reward design to encourage agreement between paired responses.

In practice

• Assess LVLMs with ConVBench's complex reasoning.
• Apply GRPO-based RL for LVLM consistency.
• Use dual rewards for response agreement.

Topics

• Large Vision-Language Models
• Visual Reasoning
• ConVBench
• ConVLM
• Reinforcement Learning
• Consistency Constraints

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, Computer Vision Engineer

Related on AIssential

• Be Consistent! Enhancing Robust Visual Reasoning in LVLMs with Consistency Constraints — LVLMs can achieve robust, consistent visual reasoning by training with logically equivalent question pairs and a dual accuracy-consistency reward.
• Faithful-MR1: Faithful Multimodal Reasoning via Anchoring and Reinforcing Visual Attention — Faithful-MR1 improves MLLM reasoning by directly anchoring visual attention and reinforcing its causal use.
• SD-MAR: Multi-image Analytical Reasoning via Synthetic Data and Reinforcement Learning — Synthetic data and reinforcement learning can significantly enhance Vision Language Models' multi-image analytical reasoning capabilities.
• Dual Adversarial Fine-tuning for Enhancing Robustness of Large Vision Language Model — A dual adversarial fine-tuning framework enhances LVLM robustness and generalizability by jointly optimizing visual and semantic signals.
• VG-CoT: Towards Trustworthy Visual Reasoning via Grounded Chain-of-Thought — VG-CoT enhances LVLM trustworthiness by grounding multi-step reasoning in explicit visual evidence through an automated pipeline.
• Leveraging Latent Visual Reasoning in Silence — Latent visual reasoning's value is in guiding learning, not in its inference-time presence.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computer Vision and Pattern Recognition.