GuysLLM at SemEval-2026 Task 5: NLI-Informed Regression for Graded Word-Sense Plausibility in Narrative Contexts

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Natural Language Processing, Data Science & Analytics · Depth: Expert, quick

Summary

GuysLLM's submission for SemEval-2026 Task 5 addresses graded word-sense plausibility estimation, a fine-grained regression task where traditional large language models (LLMs) struggle due to their discrete token-based outputs. The team reformulates this challenge as a Natural Language Inference (NLI) regression problem, adapting a DeBERTa-v3-large model with NLI pretraining and a specialized regression head to predict continuous plausibility scores from story-sense pairs. This NLI-finetuned model demonstrated superior rank correlation and better alignment with human judgments compared to baselines like BERT, vanilla DeBERTa, SmolLM variants, and even state-of-the-art LLMs using various prompting strategies. The findings highlight NLI-informed pretraining's effectiveness for narrative plausibility regression and underscore fundamental LLM limitations in word sense disambiguation.

Key takeaway

For machine learning engineers developing systems requiring fine-grained semantic reasoning or continuous plausibility scoring, you should consider NLI-informed regression. This approach, demonstrated with DeBERTa-v3-large, offers superior rank correlation and stability compared to standard LLM prompting or other baselines, mitigating issues like unstable prompting sensitivity and mean predictions. Integrating NLI pretraining can significantly improve your model's alignment with human judgments for narrative contexts.

Key insights

NLI pretraining significantly enhances LLM performance for continuous, fine-grained word-sense plausibility regression.

Principles

• NLI-informed pretraining is highly effective for narrative plausibility regression.
• LLMs exhibit fundamental limitations for word sense disambiguation tasks.

Method

Adapt DeBERTa-v3-large with NLI pretraining and a regression head to predict continuous plausibility scores from story-sense pairs, reformulating the task as NLI regression.

In practice

• Apply NLI-finetuned models for fine-grained semantic regression.
• Use NLI pretraining to improve continuous scoring tasks.

Topics

• Natural Language Inference
• Word Sense Disambiguation
• Large Language Models
• DeBERTa-v3
• Regression Tasks
• SemEval-2026

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• LLMs for automatic annotation of Mandarin narrative transcripts — LLMs can automate complex discourse-level annotation in non-English speech, significantly reducing time.
• KliniskVestBERT: BERT Model Specialised to Norwegian Clinical Texts — Domain-specific pre-training significantly enhances BERT models for Norwegian clinical NLP tasks.
• I spent 4 days training an AI model to understand my country. It barely works — Low-resource NLP projects face significant challenges in data collection, translation, and accurate labeling, often yielding modest baselines.
• Comparing BERT Sentence-Pair Classification and Few-Shot LLM Prompting for Detecting Threat and Solution Framing in German Climate News — Fine-tuned BERT models outperform few-shot LLMs for German climate news framing detection, especially with sentence context.
• Comparing LLM and Fine-Tuned Model Performance on NVDRS Circumstance Extraction with Varying Prompt Complexity — LLMs, guided by a context-aware "Complexity Score" for prompt selection, significantly outperform fine-tuned models on inferentially complex, low-prevalence data extraction.
• Do Language Models Know Theo Has a Wife? Investigating the Proviso Problem — LLMs align with human judgments on the proviso problem but use shallow pattern matching, not deep pragmatic reasoning.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.