Narrative Team at SemEval-2026 Task 5: Rating Plausibility of Word Senses in Ambiguous Sentences through Narrative Understanding

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Natural Language Processing · Depth: Expert, medium

Summary

The Narrative Team's system for SemEval-2026 Task 5 addresses the challenge of predicting word sense plausibility within ambiguous narrative contexts. This system assigns a real-valued plausibility score to candidate word senses, benchmarked against aggregated human judgments. The team explored two primary modeling approaches: a fine-tuned DistilBERT transformer-based regression model and a lightweight bidirectional LSTM neural baseline, which was either trained from scratch or initialized with GloVe embeddings. Input representations for both models combined a candidate sense definition with the narrative context and target sentence, using a special token for separation. On the official test set, the DistilBERT model achieved the strongest performance with an Acc@SD score of 0.54 and a Spearman correlation of 0.17. The best BiLSTM submission, while competitive under a tolerance-based metric, recorded an Acc@SD of 0.52 and a Spearman correlation of 0.02.

Key takeaway

For NLP Engineers developing systems for semantic evaluation, particularly word sense disambiguation in narrative contexts, you should prioritize transformer-based models like DistilBERT. While a bidirectional LSTM offers a competitive baseline with lower computational demands, DistilBERT demonstrated superior performance, achieving an Acc@SD of 0.54 and Spearman correlation of 0.17. Structure your input by clearly separating sense definitions, narrative context, and target sentences with a special token to optimize model understanding.

Key insights

Predicting word sense plausibility in ambiguous narrative contexts can be effectively modeled using transformer-based regression.

Principles

• Transformer models generally outperform recurrent networks for narrative sense disambiguation.
• Human judgments provide a robust benchmark for word sense plausibility.
• Contextual embeddings are crucial for disambiguating word senses.

Method

Combine candidate sense definition, narrative context, and target sentence with a special token. Fine-tune DistilBERT for regression on human-judged plausibility scores or train a BiLSTM.

In practice

• Employ DistilBERT for high-accuracy word sense plausibility tasks.
• Consider BiLSTM for competitive results with reduced computational overhead.
• Structure model input by separating context and definition with a special token.

Topics

• SemEval-2026 Task 5
• Word Sense Disambiguation
• Narrative Understanding
• DistilBERT
• Bidirectional LSTMs
• Semantic Evaluation

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• From TF-IDF to Transformers: Implementing Four Generations of Semantic Search — Semantic search evolved from explicit rules to learned contextual representations, balancing interpretability with semantic depth.
• SemEval-2026 Task 4: Narrative Story Similarity and Narrative Representation Learning — The NSNRL task operationalizes narrative similarity for evaluation and identifies areas for automated system improvement.
• [NLP Series] Why Words Don’t Mean Just One Thing: Ambiguity in Language — Ambiguity is a fundamental language feature that NLP systems must explicitly address across multiple linguistic levels.
• LoRi: Low-Rank Distillation for Implicit Reasoning — Hidden-state reasoning trajectories exhibit low-rank structure, enabling efficient distillation for implicit reasoning.
• Ukrainian Visual Word Sense Disambiguation Benchmark — Multimodal LLMs significantly underperform in Ukrainian Visual-WSD compared to English, revealing a critical language resource gap.
• MiqraBERT: Regression-Based Sentence-BERT Finetuning for Biblical Hebrew Parallel Detection — MiqraBERT finetunes Sentence-BERT for Biblical Hebrew semantic parallel detection, significantly improving narrative textual reuse identification over lexical methods.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.