SlugRAG at SemEval-2026 Task 8: Domain-Specific Fine-Tuning and Model Scaling for Multi-Turn RAG Retrieval

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, quick

Summary

SlugRAG at SemEval-2026 Task 8 presents a systematic evaluation of dense retrieval optimization for Multi-Turn Retrieval-Augmented Generation (MT-RAG) on the MTRAGEval benchmark (Task 8, Subtask A: Retrieval Only). The study investigated training-time strategies and inference-time query reformulation across four English-language domains: CLAPNQ (legal/patent), FIQA (financial), GOVT (government documents), and CLOUD (cloud computing). Domain-specific fine-tuning yielded the most substantial gains, with the best CLAPNQ model achieving Recall@10 of 0.6016 and nDCG@10 of 0.4981, representing 58.3% and 66.0% improvements over the pre-trained BGE baseline. Across all domains, domain-specific models averaged 44.3% improvement in Recall@10 and 47.8% in nDCG@10. Fine-tuning larger embedding models, specifically BGE-large, achieved the best overall performance (nDCG@10: 0.5101, Recall@10: 0.6221), demonstrating the combined impact of model capacity and domain adaptation.

Key takeaway

For Machine Learning Engineers developing multi-turn RAG systems, you should prioritize domain-specific fine-tuning of your dense retrieval models. This approach delivers substantial performance improvements, averaging 44.3% in Recall@10. Additionally, investigate scaling to larger embedding models, such as BGE-large, as this further enhances retrieval accuracy and overall system effectiveness for context-dependent conversational AI.

Key insights

Domain-specific fine-tuning and model scaling significantly boost multi-turn RAG retrieval performance.

Principles

• Domain-specific fine-tuning is crucial for MT-RAG performance.
• Larger embedding models enhance domain adaptation benefits.

Method

The study evaluated training-time strategies and inference-time query reformulation for dense retrieval optimization on the MTRAGEval benchmark across diverse English-language domains.

In practice

• Fine-tune dense retrievers on domain-specific data for MT-RAG.
• Consider larger embedding models like BGE-large for better results.

Topics

• Multi-Turn RAG
• Dense Retrieval
• Domain-Specific Fine-Tuning
• Embedding Models
• MTRAGEval
• SemEval-2026

Best for: Research Scientist, AI Architect, AI Engineer, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• Building HITL Feedback RAG: Embeddings, Retrieval, and Reranking — HITL Feedback RAG improves LLM accuracy by dynamically injecting human-curated corrections via a robust retrieval and reranking pipeline.
• Specializing a Small Language Model for Closed-Domain Portuguese RAG using Knowledge Graph Supervision — Fine-tuning SLMs with knowledge graph supervision significantly enhances closed-domain RAG performance, surpassing larger LLMs.
• Why Fine-Tuning Won’t Fix Your Company Data Problem — Fine-tuning is for general patterns; retrieval is for precise facts from specific documents.
• MKG-RAG-Bench: Benchmarking Retrieval in Multimodal Knowledge Graph-Augmented Generation — Retrieval quality is a critical bottleneck and strongly determines generation outcomes in multimodal knowledge graph RAG.
• Agent-Orchestrated Adaptive RAG: A Comparative Study on Structured and Multi-Hop Retrieval — Agentic RAG enhancements are domain-dependent and cost-intensive, necessitating selective, adaptive orchestration.
• Ranking Millions of Candidates with LLMs via Aggregated Embeddings, Converting Single-Vector Retrievers into Multi-Vector Models Without Retraining, and More! — Information retrieval research is rapidly advancing, focusing on LLM integration, multi-vector models, and efficiency at scale.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.