DUTIR at SemEval-2026 Task 8: A Hybrid Retrieval and Faithfulness-Guarded Framework for Multi-Turn RAG

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Expert, short

Summary

The DUTIRtaskC system, submitted for SemEval-2026 Task 8: MTRAGEval (Task C), presents a hybrid retrieval and faithfulness-guarded framework designed to tackle challenges in multi-turn Retrieval Augmented Generation (RAG). This system employs a multi-stage pipeline, beginning with GPT-5.2-powered LLM-based query rewriting to resolve conversational dependencies. It then utilizes a hybrid retrieval module that combines BGE-M3 dense embeddings and BM25 sparse retrieval, integrated via Reciprocal Rank Fusion (RRF). Further stages include a confidence-based answerability gating mechanism and a post-generation faithfulness guard to mitigate hallucinations. On the blind test set, the DUTIR system achieved a Composite Score of 0.5576, securing 4th place among 29 participating teams. Analysis indicates its strong performance in faithfulness and its ability to handle underspecified queries effectively.

Key takeaway

For Machine Learning Engineers developing multi-turn RAG systems, you should consider adopting a multi-stage pipeline approach to enhance performance and reliability. Integrating LLM-powered query rewriting, such as with GPT-5.2, can resolve conversational dependencies, while a hybrid retrieval strategy combining dense and sparse methods with RRF will boost precision. Furthermore, implementing confidence-based answerability gating and post-generation faithfulness guards is critical to mitigate hallucinations and improve overall system trustworthiness.

Key insights

A multi-stage RAG pipeline combining LLM-driven query rewriting, hybrid retrieval, and faithfulness guards significantly improves multi-turn conversational performance.

Principles

• Multi-stage RAG pipelines enhance context tracking and reduce hallucinations.
• Hybrid retrieval (dense + sparse) with RRF improves precision.
• Faithfulness guards are crucial for RAG system reliability.

Method

The proposed method involves LLM-based query rewriting (GPT-5.2), hybrid retrieval (BGE-M3, BM25, RRF), confidence-based answerability gating, and a post-generation faithfulness guard.

In practice

• Implement GPT-5.2 for conversational query rewriting in RAG.
• Combine BGE-M3 and BM25 with RRF for robust retrieval.
• Integrate answerability gating and faithfulness checks in RAG pipelines.

Topics

• Multi-turn RAG
• Hybrid Retrieval
• LLM Query Rewriting
• Faithfulness Guard
• SemEval-2026 Task 8
• GPT-5.2

Best for: AI Engineer, Research Scientist, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• RAG Is Not Dead. You’re Just Building It Wrong. — Optimizing RAG for production requires systematic engineering across all layers, prioritizing retrieval over LLM changes.
• RCEM: Robust Conversational Search EMbedder in Distributional Shift — RCEM distills LLM query rewriting into an embedder for robust, low-latency conversational search without explicit rewriting.
• Uncertainty-Aware Hybrid Retrieval for Long-Document RAG — UMG-RAG adaptively fuses multi-granularity, multi-expert retrieval based on query-specific uncertainty to create compact, coherent RAG contexts.
• Benchmarking Retrieval Strategies for Biomedical Retrieval-Augmented Generation: A Controlled Empirical Study — Cross-Encoder Reranking significantly improves biomedical RAG performance by enhancing contextual precision.
• The retrieval rebuild: Why hybrid retrieval intent tripled as enterprise RAG programs hit the scale wall — Enterprise RAG is shifting from simple vector search to complex hybrid retrieval for production agentic AI.
• Hybrid Search and Re-Ranking in Production RAG — Combining hybrid search, cross-encoder re-ranking, and metadata filtering significantly improves RAG system accuracy for enterprise knowledge.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.