Polito Team at SemEval-2026 Task 8: Scaling Multi-Turn RAG: High-Performance Parallelized Pipeline for the MTRAG Benchmark

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics, Software Development & Engineering · Depth: Expert, quick

Summary

The Polito Team developed a high-performance, parallelized Multi-Turn RAG pipeline to address the MTRAGEval task 8 at SemEval-2026. This system tackles the challenges of maintaining context and generating answers based on conversation history in multi-turn Retrieval-Augmented Generation (RAG). The pipeline is designed for three subtasks: Retrieval (Subtask A), Generation (Subtask B), and End-to-End RAG (Subtask C). Utilizing a Streamlit framework, it supports diverse corpora, various vector spaces, and embedding models. Experiments focused on the performance of different vector databases, embedding models, the role of LLM-based query rewriting (QR) for non-standalone questions, and the impact of rerankers. The team found that a configuration incorporating query rewriting alongside reranking delivered the best results. The code is publicly available on GitHub.

Key takeaway

For Machine Learning Engineers optimizing multi-turn RAG systems, prioritize integrating LLM-based query rewriting and reranking. Your system's ability to maintain conversational context and generate accurate answers significantly improves when non-standalone questions are rewritten and retrieved documents are reranked. Consider using a framework like Streamlit to flexibly configure embedding models and vector databases, enabling rapid experimentation and performance tuning for your specific corpora.

Key insights

Query rewriting and reranking are crucial for high-performance multi-turn RAG systems.

Principles

• Context maintenance is key in multi-turn RAG.
• LLM-based query rewriting improves non-standalone questions.
• Reranking enhances retrieval performance.

Method

A parallelized Multi-Turn RAG pipeline addresses Retrieval, Generation, and End-to-End RAG. It uses Streamlit to embed diverse corpora with various vector spaces and embedding models, integrating query rewriting and reranking.

In practice

• Implement LLM-based query rewriting.
• Integrate rerankers into retrieval.
• Utilize Streamlit for flexible RAG configuration.

Topics

• Multi-Turn RAG
• Query Rewriting
• Reranking
• Streamlit Framework
• Vector Databases
• LLM-based Generation

Code references

• merttoprak1/MTRAGEval-Evaluating-Multi-Turn-RAG-Conversations

Best for: AI Engineer, Research Scientist, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• Issue #115 - Reranking in your RAG pipeline — Reranking enhances RAG pipeline precision by reordering initially retrieved documents using a Cross-Encoder.
• Long Context Is Not Replacing RAG - It Is Forcing Us to Rethink Context Engineering — Long context models enhance, rather than replace, RAG, demanding advanced context engineering.
• Building HITL Feedback RAG: Embeddings, Retrieval, and Reranking — HITL Feedback RAG improves LLM accuracy by dynamically injecting human-curated corrections via a robust retrieval and reranking pipeline.
• Query-focused and Memory-aware Reranker for Long Context Processing — A new reranking framework uses attention scores for listwise relevance estimation, outperforming existing methods in long context processing.
• 10 Practical RAG Approaches: What Is Actually Useful and When to Use Each One — Effective RAG implementation requires a phased approach, starting simple and adding complexity only as specific bottlenecks are identified.
• Advanced Retrieval Pipeline for RAG (HyDE, Hybrid Search, Reranking) | Build 100% Local Retrieval — Combining HyDE, hybrid search, and reranking significantly enhances RAG retrieval accuracy and relevance.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.