ClaimDiff-RL: Fine-Grained Caption Reinforcement Learning through Visual Claim Comparison

2026-05-22 · Source: cs.LG updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, extended

Summary

The ClaimDiff-RL framework addresses the reward granularity problem in long-form image captioning reinforcement learning by introducing reference-conditioned atomic claim differences as its reward unit. Traditional holistic rewards often compress local errors into a single sequence-level signal, obscuring the critical trade-off between caption factuality and coverage. ClaimDiff-RL employs a multimodal judge to enumerate visually grounded differences between an actor caption and a reference caption, verify each difference against the image, and assign open-vocabulary error types and severity levels. These per-difference statistics are then composed into scalar rewards, allowing for separate measurement and tuning of hallucinated claims and omitted salient facts. Experiments on a 160-image human-labeled diagnostic benchmark, public captioning benchmarks, and VQA benchmarks demonstrate that ClaimDiff-RL significantly improves the balance between hallucination and missing facts, preserves general capabilities, and surpasses Gemini-3-Pro-Preview on fine-grained dimensions like object counting and spatial relations.

Key takeaway

For Machine Learning Engineers developing long-form image captioning models, you should consider adopting fine-grained reward mechanisms like ClaimDiff-RL. Your current holistic scalar rewards might inadvertently encourage conservative under-captioning, sacrificing coverage for reduced hallucination. By implementing image-verified claim differences, you can achieve a more balanced and controllable trade-off between factuality and informativeness, potentially surpassing existing models like Gemini-3-Pro-Preview on specific visual attributes. Explore severity weighting and ambiguity penalties to fine-tune your model's behavior.

Key insights

ClaimDiff-RL uses image-verified claim differences as fine-grained RL rewards for balanced, factual, and informative long-form image captions.

Principles

• Fine-grained rewards expose factuality-coverage trade-offs.
• Decomposed, image-verified claims enable explicit error tuning.
• Reference captions serve as comparison anchors, not ground truth.

Method

ClaimDiff-RL's multimodal judge identifies actor-reference visual differences, verifies them against the image, assigns typed errors (e.g., color_hallucination, count_mismatch), and composes these statistics into scalar relative or actor-only rewards for GRPO optimization.

In practice

• Use claim-difference judging for fine-grained caption RL.
• Implement severity weighting for specific error penalties.
• Apply ambiguity penalties to discourage hedging.

Topics

• Image Captioning
• Reinforcement Learning
• Multimodal LLMs
• Hallucination Detection
• Reward Design
• Visual Grounding

Code references

• ltl3A87/ClaimDiff-RL

Best for: Computer Vision Engineer, Research Scientist, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• MultiToP: Learning to Patch Visual Tokens to Mitigate Hallucinations in Video Large Multimodal Models — Hallucinations in VideoLMMs can be mitigated by selectively patching unreliable visual tokens before language generation.
• Analyzing and Improving Fine-grained Preference Optimization in Medical LVLMs — Fine-grained preference optimization improves medical LVLM accuracy by addressing token-level errors and visual grounding.
• Paying More Attention to Visual Tokens in Self-Evolving Large Multimodal Models — Self-evolving LMMs often under-condition visually; VISE uses invariance rewards to enforce spatial and semantic consistency.
• Faithful-MR1: Faithful Multimodal Reasoning via Anchoring and Reinforcing Visual Attention — Faithful-MR1 improves MLLM reasoning by directly anchoring visual attention and reinforcing its causal use.
• Mitigating Visual Hallucinations in Multimodal Systems through Retrieval-Augmented Reliability-Aware Inference — Retrieval-augmented reliability-aware inference significantly reduces visual hallucinations in MLLMs without retraining.
• DiCoBench: Benchmarking Multi-Image Fine-Grained Perception via Differential and Commonality Visual Cues — MLLMs significantly underperform humans in high-resolution, multi-image fine-grained perception, highlighting a critical research gap.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.LG updates on arXiv.org.