DeltaV: Thinking with Visual State Updates in Unified Large Multimodal Models

2026-07-09 · Source: Computer Vision and Pattern Recognition · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, quick

Summary

DeltaV is a Unified Large Multimodal Model (ULMM) that enhances interleaved multimodal reasoning by replacing traditional full-image generation with compact visual updates. Unlike existing ULMMs that generate entire images for each visual state, DeltaV incrementally predicts update tokens based on historical states, modeling only the visual changes. This approach, facilitated by a temporal similarity (TSIM) Router, reduces newly generated visual tokens by 55.6% on average without compromising reconstruction fidelity, while improving multimodal reasoning by 3.3%. The model was trained using StructCoT, a new large-scale dataset with 1.05M samples across 44 task domains. DeltaV-2B significantly outperforms larger open-source models by 8.4% on in-domain evaluations and surpasses the comparable-scale Qwen3-VL-2B by 5.9% on external benchmarks.

Key takeaway

For Machine Learning Engineers optimizing Unified Large Multimodal Models for efficiency and performance, you should consider adopting visual update paradigms like DeltaV's. This approach significantly reduces visual token generation by 55.6% and improves multimodal reasoning by 3.3% over full-image generation. Implementing such a system allows for more efficient model deployment and better performance on complex reasoning tasks, especially when leveraging diverse datasets like StructCoT.

Key insights

DeltaV improves multimodal reasoning efficiency by using compact visual state updates instead of full-image generation.

Principles

• Incremental visual updates reduce token redundancy.
• Align token budget with visual change magnitude.
• Diverse datasets enhance model generalizability.

Method

DeltaV incrementally predicts compact update tokens conditioned on historical visual states, using a temporal similarity (TSIM) Router to stop token allocation when marginal reconstruction gain falls below a threshold.

In practice

• Implement visual updates for ULMM efficiency.
• Utilize TSIM Router for dynamic token allocation.
• Develop large-scale, diverse reasoning datasets.

Topics

• Unified Large Multimodal Models
• Visual State Updates
• Multimodal Reasoning
• Token Efficiency
• StructCoT Dataset
• Temporal Similarity Router

Code references

• Pengjie-W/DeltaV

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, Computer Vision Engineer

Related on AIssential

• Visual-OPSD: Cross-Modal On-Policy Self-Distillation for Efficient Unified Multimodal Reasoning — The generation pathway of visual thoughts encodes reasoning that can be distilled for efficient, text-only multimodal models.
• Stepwise Token Selection for Efficient Multimodal Large Language Models — Stepwise token selection dynamically prunes visual tokens in MLLMs, improving efficiency by sequential, context-aware decisions.
• MMBench-Live: A Continuously Evolving Benchmark for Multimodal Models — MMBench-Live offers a scalable, automated pipeline for continuously evolving multimodal benchmarks, mitigating staleness and contamination.
• Unified Multimodal Autoregressive Modeling with Shared Context-Visual Tokenizer is Key to Unification — A single discrete visual tokenizer unifies multimodal understanding and generation in autoregressive models.
• MultiToP: Learning to Patch Visual Tokens to Mitigate Hallucinations in Video Large Multimodal Models — Hallucinations in VideoLMMs can be mitigated by selectively patching unreliable visual tokens before language generation.
• How Multimodal Large Language Models See, Think, and Reason — MLLMs integrate visual perception into transformers by converting image patches into tokens for sequential processing and reasoning.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computer Vision and Pattern Recognition.