clulab-retrieval at SemEval-2026 Task 8: A Comparative Analysis of Dense Retrievers and HyDE for Multi-Turn Conversational Retrieval

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Natural Language Processing · Depth: Expert, medium

Summary

clulab-retrieval presented a comparative analysis of dense retrievers and retrieval strategies for multi-turn conversational retrieval at SemEval-2026 Task 8 (MTRAGEval). Their official submission utilized a fine-tuned E5-based dense retriever (E5-FT, ~110M parameters) combined with Hypothetical Document Embeddings (HyDE), achieving an nDCG@5 of .3309 and ranking 31st among 38 systems. On the development set, the team compared E5-FT against BGE embeddings, dense-only versus hybrid retrieval, and HyDE versus keyword extraction. Findings indicated that the general-purpose BGE model (~110M) outperformed the domain-fine-tuned E5-FT (~110M) by 30.5% on baseline retrieval, suggesting model selection's importance over domain-specific fine-tuning. Hybrid retrieval, integrating BM25 and dense methods, offered complementary signals, with HyDE boosting BM25 by 26.7% and dense retrieval by 4.0%. Conversely, keyword-based query simplification degraded performance by 11-28%, validating HyDE's effectiveness in maintaining semantic richness.

Key takeaway

For NLP Engineers building multi-turn conversational retrieval systems, evaluate general-purpose dense retrievers like BGE early. These models can offer superior baseline performance, potentially outperforming domain-specific fine-tuning. Implement a hybrid retrieval strategy, combining BM25 with dense methods, and integrate Hypothetical Document Embeddings (HyDE). This approach significantly improves retrieval effectiveness, boosting BM25 by 26.7% and dense retrieval by 4.0%. It also avoids performance degradation from keyword simplification.

Key insights

General-purpose dense retrievers can outperform domain-fine-tuned models, and hybrid retrieval with HyDE significantly enhances multi-turn conversational search.

Principles

• General-purpose models like BGE can surpass domain-fine-tuned E5-FT by 30.5%.
• Hybrid retrieval combining BM25 and dense methods provides complementary signals.
• Keyword-based query simplification degrades performance by 11-28%.

In practice

• Prioritize general-purpose dense retrievers like BGE for multi-turn conversational retrieval.
• Implement hybrid retrieval by combining BM25 with dense methods for improved results.
• Utilize Hypothetical Document Embeddings (HyDE) to preserve semantic richness.

Topics

• Multi-turn Conversational Retrieval
• Dense Retrievers
• Hypothetical Document Embeddings
• BGE Embeddings
• Hybrid Retrieval
• BM25

Best for: AI Engineer, Machine Learning Engineer, Research Scientist, AI Scientist, NLP Engineer

Related on AIssential

• Advanced Retrieval Pipeline for RAG (HyDE, Hybrid Search, Reranking) | Build 100% Local Retrieval — Combining HyDE, hybrid search, and reranking significantly enhances RAG retrieval accuracy and relevance.
• RCEM: Robust Conversational Search EMbedder in Distributional Shift — RCEM distills LLM query rewriting into an embedder for robust, low-latency conversational search without explicit rewriting.
• Is BM25 Enough for Agentic Deep Research?, Replacing Decoders with MLPs in Generative Recommendation, and More! — Lexical retrieval with LLMs can outperform dense retrievers while multimodal embeddings can extend frozen text backbones.
• An End-to-End Hybrid Framework for Rumour Detection in Low-Resources Algerian Dialect — A hybrid framework combining transformer embeddings with classical classifiers enables effective rumour detection in low-resource Algerian dialect.
• SmartSearch: How Ranking Beats Structure for Conversational Memory Retrieval — Deterministic retrieval with intelligent ranking from unstructured data can outperform complex LLM-based memory systems.
• Not your usual search system — feat. BM25, FAISS and XGBoost — Production-grade search systems combine lexical and semantic retrieval with learning-to-rank for optimal performance.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.