UTRAG at SemEval-2026 Task 8: History-Aware Query Rewriting and LoRA-Finetuned Generation for Multi-Turn RAG

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Natural Language Processing · Depth: Expert, quick

Summary

The UTRAG system, developed by Ke Zhou and Yi-Shan Lin, addresses SemEval-2026 Task 8, "Evaluating Multi-Turn RAG Conversations (MTRAGEval)," which focuses on retrieval-augmented generation (RAG) in context-dependent, multi-turn settings. UTRAG improves retrieval through history-aware query rewriting and enhances generation faithfulness using a LoRA-adapted model, integrating these into an end-to-end pipeline. The system achieved competitive performance, with an nDCG@5 of 0.4855 in Subtask A and a harmonic mean score of 0.6554 in Subtask B, outperforming strong baselines in both. It also scored 0.5159 in Subtask C, remaining competitive. Analysis revealed that longer dialogues introduce cumulative errors in history selection and query formulation, leading to incomplete or drifting retrieval results and a higher risk of hallucination.

Key takeaway

For Machine Learning Engineers developing multi-turn RAG systems, you should integrate history-aware query rewriting to improve retrieval accuracy and consider LoRA-finetuning your generation models for enhanced faithfulness. This approach directly addresses the cumulative errors and hallucination risks that increase with dialogue length. Proactively managing conversational history and adapting generation models can significantly boost your system's reliability and performance in complex, context-dependent interactions.

Key insights

History-aware query rewriting and LoRA-finetuned generation enhance multi-turn RAG performance and faithfulness, mitigating cumulative errors.

Principles

• Multi-turn RAG needs history-aware query processing.
• LoRA adaptation boosts RAG generation faithfulness.
• Longer dialogues increase cumulative RAG errors.

Method

UTRAG employs an end-to-end pipeline integrating history-aware query rewriting for retrieval and a LoRA-adapted model for generation. This addresses multi-turn RAG challenges by improving context handling and faithfulness.

In practice

• Implement history-aware query rewriting in RAG.
• Apply LoRA finetuning for RAG generation.
• Track RAG errors with dialogue length.

Topics

• Multi-Turn RAG
• Query Rewriting
• LoRA Finetuning
• Retrieval-Augmented Generation
• Conversational AI
• SemEval-2026 Task 8

Best for: AI Engineer, Research Scientist, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• Why LLMs Fail and How RAG Makes AI Responses Smarter and More Reliable — RAG grounds LLM responses in external data, preventing hallucinations and ensuring up-to-date, accurate answers.
• A Unified Framework for Context-Aware and Relation-Aware Graph Retrieval-Augmented Generation — HyGRAG unifies context and relations in hierarchical graphs for RAG, improving multi-hop reasoning by 9.7%.
• FIDES: Faithful Inference via Deep Evidence Signals for Retrieval-Memory Conflict in RAG — FIDES resolves RAG retrieval-memory conflict by selectively applying contrastive decoding based on token-level conflict concentration.
• Rethinking the Necessity of Adaptive Retrieval-Augmented Generation through the Lens of Adaptive Listwise Ranking — Adaptive retrieval's role shifts from noise filtering for weaker LLMs to efficiency optimization for stronger ones.
• Why Retrieval-Augmented Generation Fails: A Graph Perspective — RAG failures stem from shallow, fragmented internal information flow, not just poor retrieval.
• A RAG Chatbot with Incremental Context Retrieval based on Local LLMs for Hospital Documents — Adaptive context control is crucial for reliable and efficient RAG systems using local LLMs in healthcare.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.