GeoWeaver: Grounding Visual Tokens with Geometric Evidence before Scene Reasoning

2026-05-21 · Source: Computer Vision and Pattern Recognition · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Computer Vision · Depth: Expert, quick

Summary

GeoWeaver is a pre-reasoning geometric grounding framework designed to enhance spatio-temporal reasoning in vision-language models by providing visual representations that preserve physical geometry. Unlike existing multimodal models that often treat geometric cues as a shared signal across all visual tokens, GeoWeaver addresses the need for token-adaptive geometric evidence. It constructs a multi-level geometry bank from a frozen geometry encoder and allocates the most relevant geometric abstractions to each visual token. This selected evidence is then incorporated into visual tokens via a residual grounding operation before language modeling, creating geometry-grounded representations for downstream reasoning. Evaluations on spatial reasoning benchmarks show GeoWeaver consistently improves geometry-aware reasoning while retaining general multimodal capabilities, suggesting geometry is a fundamental representational prerequisite.

Key takeaway

For Machine Learning Engineers developing vision-language models, you should prioritize integrating token-adaptive geometric grounding early in your model architecture. GeoWeaver demonstrates that treating geometry as a representational prerequisite, rather than a late-fusion auxiliary signal, significantly enhances geometry-aware reasoning. Consider implementing multi-level geometry banks and residual grounding operations to improve your model's spatial intelligence and overall multimodal capabilities. This approach can lead to more robust and accurate scene understanding.

Key insights

GeoWeaver grounds visual tokens with token-adaptive geometric evidence as a representational prerequisite for spatio-temporal reasoning.

Principles

• Geometry is a fundamental prerequisite for spatio-temporal reasoning.
• Token-adaptive geometric evidence improves visual representations.
• Geometric cues should not be a shared signal across all tokens.

Method

GeoWeaver constructs a multi-level geometry bank, performs token-adaptive geometric evidence allocation, and incorporates selected evidence via a residual grounding operation before language modeling.

In practice

• Apply token-adaptive geometry for visual grounding.
• Integrate geometry banks into multimodal architectures.

Topics

• Vision-Language Models
• Spatio-Temporal Reasoning
• Geometric Grounding
• Visual Tokens
• Multimodal AI
• Scene Understanding

Code references

• yahooo-m/GeoWeaver

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, Computer Vision Engineer

Related on AIssential

• Learning Geometric Representations from Videos for Spatial Intelligent Multimodal Large Language Models — GeoVR fundamentally restructures MLLM's internal representations for 3D spatial intelligence using 2D video and 3D foundation model distillation, without inference overhead.
• SceneGraphGrounder: Zero-Shot 3D Visual Grounding via Structured Scene Graph Matching — Zero-shot 3D visual grounding is achieved by reformulating it as structured graph matching over a reconstructed 3D scene graph.
• Reinforcing 3D Understanding in Point-VLMs via Geometric Reward Credit Assignment — Targeted reward assignment and geometric consistency verification significantly improve 3D spatial reasoning in Point-VLMs.
• Reason, Then Re-reason: Cross-view Revisiting Improves Spatial Reasoning — Revisiting spatial hypotheses with synthesized complementary views significantly enhances MLLM reasoning from egocentric videos.
• PointVG-R: Internalizing Geometric Reasoning in MLLMs for Precise Pointing Localization via Visual Chain of Thought — PointVG-R internalizes geometric reasoning in MLLMs for precise pointing localization using visual Chain-of-Thought and adaptive reinforcement learning.
• Grounded 3D-Aware Spatial Vision-Language Modeling — GR3D integrates explicit 2D, implicit 2D, and monocular 3D grounding to enhance spatial understanding in vision-language models.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computer Vision and Pattern Recognition.