CRAFT: Critic-Refined Adaptive Key-Frame Targeting for Multimodal Video Question Answering

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Emerging Technologies & Innovation · Depth: Expert, medium

Summary

CRAFT (Critic-Refined Adaptive Key-Frame Targeting) is a query-conditioned pipeline designed for grounded multi-video question answering over real-world news events. This system surfaces query-relevant evidence from heterogeneous video archives and attributes claims to their sources. CRAFT integrates dynamic keyframe selection, per-video ASR with multilingual fallback, and a hybrid critic loop to verify and repair claims. The pipeline utilizes UNLI temporal entailment, DeBERTa-v3 for cross-claim screening, and a Llama-3.2-3B adjudicator, concluding with a citation-merging stage. On MAGMaR 2026, CRAFT achieved an overall average of 0.739, reference recall of 0.810, and citation F1 of 0.635. It also performed strongly (0.823 Avg) on a WikiVideo conversion with 52 event queries, demonstrating generalization. Ablations confirm atomic claims, ASR, and the critic loop are key drivers of performance gains.

Key takeaway

For Machine Learning Engineers developing grounded video Q&A systems, CRAFT's architecture offers a robust approach to improving accuracy and source attribution. You should consider integrating a hybrid critic loop and claim-centric evidence aggregation, as these components significantly enhance performance, especially when dealing with heterogeneous video archives and complex news events. Implementing dynamic keyframe selection and multilingual ASR can further boost your system's ability to surface relevant evidence.

Key insights

CRAFT uses a critic-refined pipeline for accurate, attributable multi-video Q&A by verifying claims across sources.

Principles

• Iterative claim verification improves accuracy.
• Multimodal evidence aggregation is crucial.
• Attribution requires source identification.

Method

CRAFT combines dynamic keyframe selection, per-video ASR, and a hybrid critic loop with UNLI, DeBERTa-v3, and Llama-3.2-3B for iterative claim verification, repair, and citation merging.

In practice

• Implement a critic loop for claim verification.
• Integrate ASR for video content analysis.
• Use Llama-3.2-3B for claim adjudication.

Topics

• Multimodal Video QA
• Claim Verification
• Critic Loop
• Keyframe Selection
• ASR
• Llama-3.2-3B
• MAGMaR 2026

Code references

• bhosalems/CRAFT

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• MAGE-RAG: Multigranular Adaptive Graph Evidence for Agentic Multimodal RAG in Long-Document QA — MAGE-RAG uses an adaptive evidence graph and query-time control for efficient multimodal RAG in long documents.
• ReFree: Towards Realistic Co-Speech Video Generation via Reward-Free RL and Multilevel Speech Guidance — ReFree-S2V balances precise lip-sync and expressive motion in co-speech video generation using multi-level speech guidance and reward-free RL.
• TimeProVe: Propose, then Verify for Efficient Long Video Temporal Reasoning in Activities of Daily Living — TimeProVe efficiently reasons in long videos by proposing hypotheses with lightweight LLMs and verifying them with targeted VLM calls.
• Contestable Multi-Agent Debate with Arena-based Argumentative Computation for Multimedia Verification — A multi-agent framework uses structured argumentation and multimodal LLMs for transparent multimedia verification.
• HarmVideoBench: Benchmarking Harmful Video Understanding in Large Multimodal Models — Harmful video understanding in LVLMs requires multi-layered evaluation beyond binary classification and explicit rationales.
• CRAFT: A Unified Counterfactual Reasoning Framework for Tabular Question Answering and Fact Verification — CRAFT uses bidirectional counterfactual reasoning to improve LLM performance on complex tabular data tasks.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.