HiKEY: Hierarchical Multimodal Retrieval for Open-Domain Document Question Answering

2026-05-28 · Source: Artificial Intelligence · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Natural Language Processing · Depth: Expert, quick

Summary

HiKEY is a hierarchical tree-based multimodal retrieval framework designed to overcome critical bottlenecks in retrieval-augmented generation (RAG) for open-domain document question answering (ODQA) on large industrial corpora. It addresses routing failure in locating correct documents and evidence fragmentation from scattered information like tables and figures. Instead of simple chunking, HiKEY reconstructs a logical heterogeneous graph using Document Hierarchical Parsing (DHP) to explicitly encode parent-child relationships. The framework employs a hierarchical coarse-to-fine strategy, first performing global routing with hierarchical indexing to prune the search space, then conducting fine-grained retrieval by ranking sections via multimodal fusion. Finally, it assembles a token-efficient evidence subgraph using a hybrid structural-semantic packing strategy. Experiments show HiKEY improves retrieval recall by up to 12.9% and end-to-end QA performance by up to 6.8% over page- and chunk-based baselines.

Key takeaway

For NLP Engineers building retrieval-augmented generation systems for complex document question answering, HiKEY offers a significant advancement. You should consider adopting hierarchical document parsing and multimodal fusion strategies to overcome routing failures and evidence fragmentation. This approach can notably improve your system's retrieval recall and overall QA performance, especially when dealing with large industrial corpora containing diverse data types like tables and figures.

Key insights

HiKEY uses document hierarchy and multimodal fusion for efficient, accurate RAG in ODQA.

Principles

• Document hierarchy improves retrieval precision.
• Multimodal fusion enhances evidence integration.
• Coarse-to-fine retrieval prunes search space.

Method

HiKEY reconstructs a logical heterogeneous graph via DHP, then uses hierarchical indexing for global routing, followed by multimodal fusion for fine-grained section ranking, and finally structural-semantic packing for evidence subgraph assembly.

In practice

• Apply DHP for complex document structures.
• Integrate multimodal data for richer context.
• Use hierarchical indexing for large corpora.

Topics

• Retrieval-Augmented Generation
• Multimodal Retrieval
• Document Question Answering
• Hierarchical Indexing
• Document Parsing
• Open-Domain QA

Best for: Research Scientist, AI Engineer, AI Scientist, NLP Engineer, Machine Learning Engineer

Related on AIssential

• Evaluation of Chunking Strategies for Effective Text Embedding in Low-Resource Language on Agricultural Documents — Structural preservation in text chunking significantly outperforms linguistic or semantic methods for effective retrieval in low-resource, technical documents.
• Scaling Retrieval-Augmented Reasoning with Parallel Search and Explicit Merging — MultiSearch uses parallel multi-query retrieval and explicit merging to enhance LLM reasoning and signal-to-noise ratio.
• Enhancing Retrieval via Cognitively Motivated Document Expansion — Cognitively motivated document expansion aims to improve information retrieval.
• Beyond Keyword Alerts: Building a Semantic PR Opportunity Monitor for a Real Agency — Semantic retrieval and RAG effectively identify proactive PR opportunities beyond traditional keyword matching.
• An Efficient Approach for Answering Not Readily Attainable Questions for RAG-based Applications — This paper proposes an efficient approach for RAG systems to answer questions not readily found in retrieved data.
• Not your usual search system — feat. BM25, FAISS and XGBoost — Production-grade search systems combine lexical and semantic retrieval with learning-to-rank for optimal performance.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Artificial Intelligence.