uva-irlab-conv at SemEval-2026 Task 8: Multi-Turn RAG with Learned Sparse Retrieval and Listwise Reranking

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, quick

Summary

The uva-irlab-conv system participated in SemEval-2026 Task 8, focusing on multi-turn retrieval and question answering across finance, cloud documentation, government, and Wikipedia domains. Their proposed multi-turn retrieval-augmented generation (RAG) pipeline integrates learned sparse retrieval with LLM-based reranking and generation. This approach leverages sparse retrieval's strong generalization capabilities and utilizes LLMs' long-context features for conversational query rewriting, pointwise and listwise reranking, and final response generation. Each step is conditioned on the full conversational history, enabling effective context integration and improving robustness, even for unanswerable queries where evidence is insufficient.

Key takeaway

For NLP Engineers developing multi-turn conversational AI, this multi-step RAG pipeline offers a robust solution for handling diverse domains and unanswerable queries. By combining learned sparse retrieval with LLM-based reranking and generation, conditioned on full conversational history, you can significantly improve system accuracy and generalization. Consider integrating these techniques to enhance your conversational systems' ability to manage complex, context-dependent interactions effectively.

Key insights

A multi-step RAG pipeline effectively integrates conversational context for robust multi-turn question answering across diverse domains.

Principles

• Sparse retrieval generalizes well across domains.
• LLMs enhance conversational context integration.
• Multi-step RAG improves system robustness.

Method

A multi-turn RAG pipeline employs learned sparse retrieval, followed by LLM-based conversational query rewriting, pointwise and listwise reranking, and final response generation, all conditioned on the full conversational history.

In practice

• Combine sparse retrieval with LLM reranking.
• Use LLMs for conversational query rewriting.
• Condition generation on full chat history.

Topics

• Multi-Turn RAG
• Sparse Retrieval
• LLM Reranking
• Conversational AI
• SemEval-2026
• Question Answering

Best for: AI Engineer, Machine Learning Engineer, Research Scientist, AI Scientist, NLP Engineer

Related on AIssential

• RCEM: Robust Conversational Search EMbedder in Distributional Shift — RCEM distills LLM query rewriting into an embedder for robust, low-latency conversational search without explicit rewriting.
• Benchmarking Retrieval Strategies for Biomedical Retrieval-Augmented Generation: A Controlled Empirical Study — Cross-Encoder Reranking significantly improves biomedical RAG performance by enhancing contextual precision.
• SmartSearch: How Ranking Beats Structure for Conversational Memory Retrieval — Deterministic retrieval with intelligent ranking from unstructured data can outperform complex LLM-based memory systems.
• The State of Information Retrieval in 2026 — Information retrieval has evolved into a reasoning problem, integrating LLMs and agentic search for complex, context-aware query processing.
• An Efficient Approach for Answering Not Readily Attainable Questions for RAG-based Applications — This paper proposes an efficient approach for RAG systems to answer questions not readily found in retrieved data.
• Issue #115 - Reranking in your RAG pipeline — Reranking enhances RAG pipeline precision by reordering initially retrieved documents using a Cross-Encoder.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.