MindFlayer at SemEval-2026 Task 8:DUALRAG:Answerability-Aware Generation for Multi-Turn RAG Conversations

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Natural Language Processing · Depth: Expert, quick

Summary

The DualRAG system, developed by team MindFlayer, achieved 8th place out of 26 teams at SemEval-2026 Task 8 Subtask B for generating faithful responses in multi-turn RAG conversations. Its core innovation is an answerability-aware generation approach: it first verifies if reference passages exist for a question. If passages are available, it uses a domain-guided prompt with Meta's Llama-4-Scout-17B via the Groq API to answer strictly from those passages. If no passages are found, it employs a strict refusal prompt to decline answering, preventing hallucination. This zero-shot prompting method, requiring no training, included a lightweight post-processing layer that corrected only 7 out of 507 test tasks. DualRAG achieved a harmonic mean of 0.7492, significantly outperforming the GPT-OSS-120B baseline (0.639). Notably, it demonstrated 100% refusal accuracy on all 130 unanswerable questions, a feat even GPT-4o and Llama 3.1 405B struggled with. Its RLF score was 0.8782, indicating strong grounding.

Key takeaway

For NLP Engineers building multi-turn RAG systems, integrating an answerability check is crucial. Your system can achieve high faithfulness and refusal accuracy by routing questions to either a passage-grounded generation prompt or a strict refusal prompt based on passage availability. This approach, demonstrated with Llama-4-Scout-17B, significantly reduces hallucinations, even outperforming larger models like GPT-4o in handling unanswerable queries. Consider implementing this dual-path prompting strategy to enhance your RAG system's reliability and user trust.

Key insights

Routing based on answerability prevents hallucination and improves refusal accuracy in RAG systems.

Principles

• Prioritize answerability checks before generation.
• Use distinct prompts for answering versus refusing.
• Zero-shot prompting can achieve strong RAG performance.

Method

A two-path prompting system: one path for passage-grounded answers, another for polite refusal when no passages exist, with minimal post-processing.

In practice

• Implement an answerability check before RAG generation.
• Design specific refusal prompts for unanswerable queries.
• Consider zero-shot Llama-4-Scout-17B for RAG tasks.

Topics

• RAG Systems
• Multi-Turn Conversations
• Answerability Detection
• Zero-Shot Prompting
• Llama-4-Scout-17B
• SemEval-2026

Best for: Research Scientist, AI Architect, AI Engineer, AI Scientist, NLP Engineer, Machine Learning Engineer

Related on AIssential

• Reducing Hallucinations in Complex Question Answering using Simple Graph-based Retrieval-Augmented Generation (long version) — Integrating lightweight knowledge graphs into RAG systems significantly reduces LLM hallucination and improves complex QA accuracy.
• MORTAR: Multi-turn Metamorphic Testing for LLM-based Dialogue Systems — MORTAR offers a bias-free, cost-effective metamorphic testing approach for multi-turn LLM dialogue systems.
• Building a RAG System That Knows When It’s Wrong — A robust RAG system must know when to refuse answers to prevent confident, silently false generations.
• Derivation Prompting: A Logic-Based Method for Improving Retrieval-Augmented Generation — Derivation Prompting uses logic-inspired rule application to enhance RAG generation, reducing LLM hallucinations and improving interpretability.
• FlowRAG: Synergizing Explicit Reasoning via Frequency-Aware Multi-Granularity Graph Flow — FlowRAG improves GraphRAG by integrating multi-granularity graphs and frequency-aware flow for robust semantic recall and explicit reasoning.
• TA-RAG: Tone-Aware Retrieval-Augmented Generation for Peer-Support Health Communication — TA-RAG enhances RAG with prompt-based tone control for sensitive health communication, improving empathy and readability without fine-tuning.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.