Howard University-AI4PC at SemEval-2026 Task 8: Query Reformulation and Dense-Lexical Retrieval Fusion for Multi-Turn Retrieval-Augmented Generation

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Natural Language Processing, Data Science & Analytics · Depth: Expert, medium

Summary

Howard University-AI4PC presented a training-free hybrid retrieve-then-rerank system for multi-turn retrieval-augmented generation (RAG) at SemEval-2026 Task 8 (MTRAGEval). This system addresses challenges like non-standalone questions and unanswerable queries across ClapNQ, Cloud, FiQA, and Govt. corpora. It reformulates queries using LLM-driven rewriting, decomposition, and Hypothetical Document Embeddings (HyDE). Retrieved candidates from BGE-base-en-v1.5 dense vector search and BM25 lexical matching are fused via Reciprocal Rank Fusion, then reranked by BGE-reranker-large. Llama-3.3-70B-Instruct generates extractive, context-grounded responses with built-in abstention. Without fine-tuning, the system achieved nDCG@5 of 0.4098 on Task A (22nd/38), a harmonic mean of 0.7462 on Task B (9th/26), and 0.5796 on Task C (2nd/29), coming within 1.1% of the top submission.

Key takeaway

For Machine Learning Engineers building multi-turn RAG systems, this work demonstrates that competitive performance is achievable without fine-tuning by integrating advanced query reformulation and a hybrid retrieval approach. You should consider implementing LLM-driven query rewriting, decomposition, and HyDE, alongside fusing dense and lexical search results. This strategy, combined with an extractive generator like Llama-3.3-70B-Instruct that supports abstention, can significantly improve end-to-end RAG effectiveness, especially for complex or unanswerable queries.

Key insights

A training-free hybrid RAG system combines multi-signal query reformulation and fused retrieval for competitive multi-turn performance.

Principles

• Hybrid retrieval (dense + lexical) improves recall.
• LLM-driven query reformulation enhances relevance.
• Extractive generation with abstention boosts faithfulness.

Method

Queries are reformulated via LLM rewriting, decomposition, and HyDE. Dense (BGE-base-en-v1.5) and lexical (BM25) results are fused with Reciprocal Rank Fusion, reranked by BGE-reranker-large, then Llama-3.3-70B-Instruct generates responses.

In practice

• Implement LLM-driven query rewriting for complex queries.
• Combine dense and lexical search for robust retrieval.
• Incorporate abstention for unanswerable RAG queries.

Topics

• Retrieval-Augmented Generation
• Multi-Turn RAG
• Query Reformulation
• Hybrid Retrieval
• Large Language Models
• SemEval-2026

Best for: AI Engineer, Research Scientist, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• How Retrieval Systems Power Modern RAG Applications — RAG combines external retrieval with LLMs to overcome knowledge cutoffs and hallucinations, providing grounded, up-to-date responses.
• Build a Hybrid RAG System with FAISS, BM25, LangGraph and Claude Sonnet Model — Hybrid RAG combines semantic and keyword search via Reciprocal Rank Fusion to enhance retrieval accuracy for diverse query types.
• Beyond Semantic Search: The Complete Guide to Advanced RAG with Milvus | Sagar Patil — Advanced RAG requires multi-stage retrieval, combining hybrid search with techniques like query rewriting and reranking for production reliability.
• Understanding the Behaviors of Environment-aware Information Retrieval — LLMs can learn to adapt query strategies for diverse retrievers using RL, revealing distinct optimal query styles.
• 10 Practical RAG Approaches: What Is Actually Useful and When to Use Each One — Effective RAG implementation requires a phased approach, starting simple and adding complexity only as specific bottlenecks are identified.
• Rethinking the Necessity of Adaptive Retrieval-Augmented Generation through the Lens of Adaptive Listwise Ranking — Adaptive RAG's role shifts from noise mitigation for weaker LLMs to efficiency optimization for stronger ones.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.