MARQUIS: A Three-Stage Pipeline for Video Retrieval-Augmented Generation

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Multimodal AI · Depth: Expert, quick

Summary

MARQUIS is a three-stage pipeline designed to enhance video Retrieval-Augmented Generation (RAG) by addressing limitations in retrieving relevant audiovisual evidence and synthesizing coherent, attributed text from large video corpora. The pipeline incorporates (1) query expansion, fusion, and reranking, (2) calibrated structured evidence extraction, and (3) article generation from the extracted evidence, optionally controlled by a RLM. On the MAGMaR2026 shared task, MARQUIS significantly improved retrieval performance from 0.195 to 0.759 (nDCG@10). For article generation, the ITER-QA-BASE component boosted the average human score from 3.09 to 3.83 over the CAG baseline, while MARQUIS-RLM achieved a human score of 3.30 and demonstrated the strongest citation recall among non-QA systems.

Key takeaway

For AI Scientists and Machine Learning Engineers developing video RAG systems, MARQUIS offers a robust framework to overcome current challenges with complex queries and multi-video synthesis. Its three-stage pipeline, featuring query expansion and structured evidence extraction, demonstrably boosts retrieval performance and generation quality. You should consider adopting a similar multi-stage approach to enhance your system's ability to synthesize attributed text from large video datasets.

Key insights

A three-stage pipeline significantly improves video RAG by enhancing complex query retrieval and multi-video context generation.

Principles

• Complex queries require multi-faceted retrieval methods.
• High-level reasoning is crucial for multi-video synthesis.
• Memory constraints limit long, multi-video contexts.

Method

The pipeline involves query expansion/fusion/reranking, calibrated structured evidence extraction, and RLM-controlled article generation from the extracted evidence.

In practice

• Implement query expansion for complex video queries.
• Utilize structured evidence extraction for better generation.
• Consider RLM for controlled article generation.

Topics

• Video Retrieval-Augmented Generation
• Query Expansion
• Evidence Extraction
• Multimodal Retrieval
• Natural Language Generation
• MAGMaR 2026

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• Rethinking RAG in Long Videos: What to Retrieve and How to Use It? — Optimal video RAG requires chunk-level modality and granularity decisions, not query-level.
• Decoupling Semantics and Logic: A Training-Free Coarse-to-Fine Pipeline for Video Retrieval-Augmented Generation — Decoupling semantic retrieval from logical reasoning significantly enhances Video RAG precision and persona adherence.
• Vortex: Multi-Modal Fusion System for Intelligent Video Retrieval — Vortex fuses CLIP and SigLIP2 embeddings with relevance feedback for effective multimodal video retrieval.
• From Scenes to Elements: Multi-Granularity Evidence Retrieval for Verifiable Multimodal RAG — Element-level visual evidence retrieval significantly enhances multimodal RAG verifiability and performance.
• Rethinking RAG in Long Videos: What to Retrieve and How to Use It? — VideoRAG performance improves by using chunk-adaptive retrieval and reranking, addressing limitations of existing benchmarks and single-configuration approaches.
• DCD: Domain-Oriented Design for Controlled Retrieval-Augmented Generation — DCD improves RAG accuracy and robustness by structuring knowledge hierarchically and controlling query processing without modifying the LLM.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.