Clutch or Cry at SemEval-2026 Task 12: Offline Retrieval-Augmented Generation with Frozen DeBERTa for Abductive Event Reasoning

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Natural Language Processing · Depth: Expert, medium

Summary

Prasad et al. present a two-stage offline Retrieval-Augmented Generation (RAG) pipeline for SemEval-2026 Task 12 on abductive event reasoning. This system addresses issues encountered with direct fine-tuning of large language models, which suffered from severe overfitting due to limited training data, and smaller models, which failed under context-length constraints. Their approach employs an all-MiniLM-L6-v2 dense retriever to extract the single most relevant sentence (top-k=1). This retrieved evidence is then fed into a partially frozen DeBERTa-v3-Large classifier, trained using BCEWithLogitsLoss. By freezing the lower 12 layers of DeBERTa-v3-Large, the system effectively mitigates overfitting while retaining pre-trained semantic knowledge. This method successfully eliminates long-context truncation problems, reduces hallucination, and achieved an Exact Match accuracy of 0.72 on the official test set.

Key takeaway

For Machine Learning Engineers developing abductive reasoning systems with limited data, consider adopting a two-stage offline RAG pipeline. Your approach should involve a dense retriever for evidence extraction and a partially frozen DeBERTa-v3-Large classifier. Freezing the lower 12 layers can prevent overfitting and manage context-length constraints, potentially improving Exact Match accuracy and reducing hallucination in your models.

Key insights

A two-stage RAG pipeline with a partially frozen DeBERTa-v3-Large classifier effectively solves abductive event reasoning challenges by mitigating overfitting and context issues.

Principles

• Freezing lower model layers mitigates overfitting.
• Separate retrieval from classification in RAG.
• Limited data causes large language model overfitting.

Method

A two-stage offline RAG pipeline: first, all-MiniLM-L6-v2 retrieves top-k=1 relevant sentence; second, a partially frozen DeBERTa-v3-Large classifier performs multi-label classification using BCEWithLogitsLoss.

In practice

• Use dense retriever for evidence extraction.
• Freeze lower layers to prevent overfitting.
• Apply RAG for context-length issues.

Topics

• Retrieval-Augmented Generation
• Abductive Event Reasoning
• DeBERTa-v3-Large
• Model Overfitting
• Semantic Evaluation
• all-MiniLM-L6-v2

Best for: AI Engineer, Research Scientist, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• AILS-NTUA at SemEval-2026 Task 12: Graph-Based Retrieval and Reflective Prompting for Abductive Event Reasoning — AILS-NTUA's three-stage system won SemEval 2026 Task 12 for Abductive Event Reasoning, achieving a first-place rank with an accuracy score of 0.95.
• Retrieval-Augmented Generation — A Deep Dive into Components — RAG improves LLM accuracy and relevance by integrating external knowledge retrieval with text generation.
• A Four-Condition Diagnostic Protocol for Evidence Utilization in Long-Context and Retrieval-Augmented Language Models — Traditional metrics are insufficient; a four-condition protocol is crucial for diagnosing true evidence utilization in LLMs.
• When Cases Get Rare: A Retrieval Benchmark for Off-Guideline Clinical Question Answering — LLMs require retrieval augmentation and strong reasoning for reliable off-guideline medical question answering.
• Fast and Faithful: Real-Time Verification for Long-Document Retrieval-Augmented Generation Systems — Full-context verification for RAG systems significantly improves hallucination detection in long documents under real-time constraints.
• A Unified Framework for Context-Aware and Relation-Aware Graph Retrieval-Augmented Generation — HyGRAG unifies context and relations in hierarchical graphs for RAG, improving multi-hop reasoning by 9.7%.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.