MARDoc: A Memory-Aware Refinement Agent Framework for Multimodal Long Document QA

2026-06-04 · Source: Takara TLDR - Daily AI Papers · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems · Depth: Expert, medium

Summary

MARDoc is a Memory-Aware Refinement Agent framework designed for multimodal long-document question answering (QA). It addresses the issue in existing iterative retrieval-reasoning agents where a single, growing context mixes retrieval traces, observations, and intermediate reasoning, leading to scattered evidence and noisy multi-hop reasoning. MARDoc decouples the QA process into three specialized agents: an Explorer for multi-granularity multimodal retrieval, a Refiner for distilling interaction traces into structured evidence and reasoning memories, and a Reflector for checking evidence sufficiency and providing targeted feedback. These agents utilize a dynamically updated structured memory instead of a full accumulated interaction history, which reduces context noise while preserving critical facts and their logical dependencies. Experiments on MMLongBench-Doc and DocBench demonstrate MARDoc's strong performance, outperforming same-backbone baselines and validating the effectiveness of its structured memory approach for agentic document QA.

Key takeaway

For Machine Learning Engineers developing multimodal long-document QA systems, MARDoc's agentic framework offers a clear path to improve performance. You should consider decoupling your QA pipeline into specialized agents for retrieval, refinement, and reflection. Implementing a dynamically updated structured memory, rather than a monolithic context, will significantly reduce noise and preserve critical evidence, leading to more accurate multi-hop reasoning. This approach can enhance your system's ability to handle complex, lengthy documents effectively.

Key insights

MARDoc improves long-document QA by using specialized agents and structured memory to reduce context noise and preserve critical evidence.

Principles

• Decouple complex QA into specialized agents.
• Structured memory reduces context noise.
• Dynamically update memory over full history.

Method

MARDoc employs an Explorer for multimodal retrieval, a Refiner for distilling traces into structured memories, and a Reflector for feedback, all relying on a dynamically updated structured memory.

In practice

• Implement specialized agents for QA tasks.
• Design structured memory for evidence.

Topics

• Multimodal QA
• Long Document Processing
• Agent Frameworks
• Structured Memory
• Retrieval-Reasoning
• MARDoc

Best for: Research Scientist, AI Engineer, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• MARDoc: A Memory-Aware Refinement Agent Framework for Multimodal Long Document QA — MARDoc uses specialized agents and structured memory to refine multimodal long-document QA, reducing noise and preserving critical evidence.
• MARDoc: A Memory-Aware Refinement Agent Framework for Multimodal Long Document QA — MARDoc uses a structured, dynamically updated memory to overcome context dilution in multimodal long-document QA agents.
• MemoryDocDataSet: A Benchmark for Joint Conversational Memory and Long Document Reasoning — MemoryDocDataSet uniquely benchmarks AI systems on joint conversational memory and long document reasoning, revealing a critical performance gap.
• AMA: Adaptive Memory via Multi-Agent Collaboration — AMA uses multi-agent collaboration and hierarchical memory to provide adaptive, consistent long-term memory for LLM agents.
• DeferMem: Query-Time Evidence Distillation via Reinforcement Learning for Long-Term Memory QA — DeferMem improves LLM long-term memory QA by decoupling retrieval from query-conditioned evidence distillation using RL.
• A-MAR: Agent-based Multimodal Art Retrieval for Fine-Grained Artwork Understanding — Explicit reasoning plans improve multimodal art retrieval, enabling grounded, interpretable artwork understanding.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Takara TLDR - Daily AI Papers.