Less is More: Controlled Visual Evidence Routing and Redundancy Compression for Key Information Extraction

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Expert, quick

Summary

The OTCR (Optimal Transport for Controlled Routing) framework addresses challenges in Key Information Extraction (KIE) from visually-rich documents, where current multimodal encoders indiscriminately fuse dense visual patches. This common practice introduces low-density visual noise, intensifies modality competition, and leads to robustness collapse under distribution shifts. OTCR, a lightweight and architecture-agnostic solution, transforms vision into a selective supporter by learning sparse, interpretable cross-modal coupling via optimal transport, routing local visual evidence to the most relevant text tokens. It further employs token-level gating to control injection strength and a variational information bottleneck to suppress spurious correlations. Experiments demonstrate consistent performance gains when OTCR is integrated with LayoutLMv3 and GeoLayoutLM on FUNSD, CORD, and SROIE datasets, and it significantly mitigates performance degradation under distribution shifts like Do-GOOD and EC-FUNSD.

Key takeaway

For Machine Learning Engineers developing Key Information Extraction (KIE) systems for visually-rich documents, especially those facing robustness issues under distribution shifts, you should re-evaluate current dense visual fusion strategies. OTCR provides a lightweight, architecture-agnostic framework that transforms vision into a selective supporter, mitigating noise and modality competition. Consider integrating OTCR with models like LayoutLMv3 or GeoLayoutLM to achieve consistent performance gains and significantly improve system resilience against real-world data variability.

Key insights

OTCR selectively routes visual evidence to text tokens, improving Key Information Extraction robustness and performance by mitigating noise and competition.

Principles

• Indiscriminate visual fusion harms KIE robustness under distribution shifts.
• Sparse, controlled visual evidence acts as a selective supporter for KIE.
• Optimal transport enables interpretable cross-modal coupling for KIE.

Method

OTCR learns sparse cross-modal coupling via optimal transport, applies token-level gating to control injection strength, and suppresses spurious correlations using a variational information bottleneck.

In practice

• Integrate OTCR with LayoutLMv3 for enhanced KIE.
• Integrate OTCR with GeoLayoutLM for improved KIE.
• Apply OTCR to mitigate KIE performance degradation under distribution shifts.

Topics

• Key Information Extraction
• Multimodal Learning
• Optimal Transport
• Variational Information Bottleneck
• LayoutLMv3
• Distribution Shift Robustness

Best for: AI Engineer, Research Scientist, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• Lost in a Single Vector: Improving Long-Document Retrieval with Chunk Evidence Aggregation — Improving long-document retrieval by aggregating independently encoded chunk-level evidence.
• OTCHA: Optimal Transport-driven Confidence-aware Latent Hub Alignment for Multi-View Medical Image Classification — OTCHA refines multi-view medical image tokens using confidence-aware optimal transport and shared latent hubs to prevent fusion contamination.
• KVCapsule: Efficient Sequential KV Cache Compression for Vision-Language Models with Asymmetric Redundancy — KVCapsule efficiently compresses VLM KV caches by leveraging asymmetric redundancy and dynamic attention patterns in vision tokens.
• Granite 4.0 3B Vision: Compact Multimodal Intelligence for Enterprise Documents — Granite 4.0 3B Vision offers compact, modular multimodal intelligence for enterprise document understanding.
• You Can't Patch a Running Plant: How Mythos Compresses the OT Security Timeline — AI-driven vulnerability discovery drastically accelerates cyberattack timelines, demanding urgent, tailored security responses for operational technology.
• Switch-KD: Visual-Switch Knowledge Distillation for Vision-Language Models — Switch-KD unifies VLM knowledge transfer in a shared text-probability space for efficient deployment.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.