CuriosAI at SemEval-2026 Task 8: Hybrid retrieval system with repeated sampling for generation

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering · Depth: Advanced, quick

Summary

The CuriosAI system, developed by Aiswariya Manoj Kumar et al., addresses multi-turn Retrieval-Augmented Generation (RAG) challenges in SemEval-2026 Task 8 (MTRAGEval), which focuses on conversational issues like non-standalone turns and underspecification. Standard single-turn RAG pipelines struggle with these complexities, leading to amplified errors. CuriosAI integrates contextual query rewriting, heterogeneous hybrid retrieval with Reciprocal Rank Fusion (RRF), domain-adaptive Low-Rank Adaptation (LoRA) reranking, and repeated sampling with metric-guided selection. This approach significantly outperformed organizers' baselines on the official test set, achieving an nDCG@5 of 0.5396 for Retrieval (vs. 0.4795), 0.7571 for Generation (vs. 0.6390), and 0.5486 for RAG (vs. 0.5366). The system ranked 5th in Subtask A, 5th in Subtask B, and 7th in Subtask C, demonstrating the effectiveness of calibrated hybrid retrieval and robust generation selection for multi-turn RAG.

Key takeaway

For Machine Learning Engineers developing conversational AI systems, this research highlights critical strategies for robust multi-turn Retrieval-Augmented Generation. You should integrate contextual query rewriting and heterogeneous hybrid retrieval, potentially using Reciprocal Rank Fusion, to address complex conversational challenges. Consider domain-adaptive LoRA reranking and repeated sampling with metric-guided selection to significantly improve your system's retrieval and generation performance, moving beyond standard single-turn RAG limitations.

Key insights

Calibrated hybrid retrieval and robust generation selection significantly improve multi-turn RAG performance in conversational AI.

Principles

• Multi-turn RAG requires specialized handling for conversational context.
• Hybrid retrieval fused with RRF enhances document relevance.
• Repeated sampling with metric-guided selection improves generation robustness.

Method

The system combines contextual query rewriting, RRF-fused hybrid retrieval, LoRA reranking, and repeated sampling with metric-guided selection to enhance multi-turn RAG.

In practice

• Implement contextual query rewriting for conversational RAG.
• Utilize Reciprocal Rank Fusion for diverse retrieval results.
• Apply LoRA for domain-adaptive reranking in RAG pipelines.

Topics

• Multi-turn RAG
• Hybrid Retrieval
• Reciprocal Rank Fusion
• LoRA Reranking
• Conversational AI
• SemEval-2026

Best for: Research Scientist, AI Architect, AI Engineer, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• Rethinking the Necessity of Adaptive Retrieval-Augmented Generation through the Lens of Adaptive Listwise Ranking — Adaptive retrieval's role shifts from noise filtering for weaker LLMs to efficiency optimization for stronger ones.
• Build a Hybrid RAG System with FAISS, BM25, LangGraph and Claude Sonnet Model — Hybrid RAG combines semantic and keyword search via Reciprocal Rank Fusion to enhance retrieval accuracy for diverse query types.
• HybridRAG: A Practical LLM-based ChatBot Framework based on Pre-Generated Q&A over Raw Unstructured Documents — HybridRAG pre-generates QA pairs from unstructured documents to enhance RAG chatbot speed and accuracy.
• A Unified Framework for Context-Aware and Relation-Aware Graph Retrieval-Augmented Generation — HyGRAG unifies context and relations in hierarchical graphs for RAG, improving multi-hop reasoning by 9.7%.
• MemoryDocDataSet: A Benchmark for Joint Conversational Memory and Long Document Reasoning — MemoryDocDataSet uniquely benchmarks AI systems on joint conversational memory and long document reasoning, revealing a critical performance gap.
• MADRAG: Multi-Agent Debate with Retrieval-Augmented Generation for Training-Free Analytic Essay Scoring — Multi-agent debate with retrieval augmentation offers a training-free, reliable approach to analytic essay scoring, outperforming prompt-based LLMs.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.