The LSCD Benchmark: a Testbed for Diachronic Word Meaning Tasks

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Expert, medium

Summary

The LSCD Benchmark repository standardizes evaluation for Lexical Semantic Change Detection (LSCD), a complex, lemma-level task. LSCD typically involves modular steps: Word-in-Context (WiC) labeling, Word Sense Induction (WSI) to form sense clusters, and then comparing these clusters over time. This modularity, combined with diverse modeling options, dataset versions, and evaluation metrics, creates significant heterogeneity, hindering comparable model evaluation, optimal model combination, and result reproduction. The benchmark offers transparent implementation for reproducibility and allows free combination of different components. It supports modular evaluation for WiC, WSI, and LSCD, facilitating careful assessment of complex model components and new optimization strategies. Presented at the 15th Joint Conference on Lexical and Computational Semantics (*SEM 2026) in July 2026, pages 148–163, the benchmark has already been used to conduct experiments with recent models and systematically improve performance.

Key takeaway

For NLP Engineers developing or evaluating Lexical Semantic Change Detection (LSCD) models, you should adopt the LSCD Benchmark. This repository offers a standardized framework to overcome current evaluation heterogeneity, ensuring your results are comparable and reproducible. Utilize its modularity to evaluate Word-in-Context and Word Sense Induction components independently, allowing for more precise model optimization and the free combination of different approaches. This will streamline your development process and help systematically improve model performance.

Key insights

The LSCD Benchmark standardizes evaluation for Lexical Semantic Change Detection, improving reproducibility and model optimization.

Principles

• Modular tasks require modular evaluation.
• Standardization improves reproducibility.
• Transparent implementation aids optimization.

Method

The benchmark operationalizes LSCD evaluation by allowing separate assessment of Word-in-Context (WiC) labeling, Word Sense Induction (WSI), and the final LSCD comparison, enabling component-level optimization.

In practice

• Evaluate WiC and WSI components independently.
• Combine different LSCD model components.
• Reproduce LSCD research results easily.

Topics

• Lexical Semantic Change Detection
• Word-in-Context
• Word Sense Induction
• NLP Benchmarking
• Computational Semantics
• Diachronic Linguistics

Best for: Research Scientist, AI Scientist, NLP Engineer

Related on AIssential

• The BD-LSC Dataset: Facilitating the Benchmarking of Models for Lexical Semantic Change Detection in Slang and Standard Usage — New benchmarks address bi-directional lexical semantic change and slang usage, revealing challenges with rare slang senses.
• Semantic Reranking at Inference Time for Hard Examples in Rhetorical Role Labeling — RISE improves Rhetorical Role Labeling on hard examples by semantically reranking low-confidence predictions at inference time.
• A Structured Benchmark for Text-Guided Anomaly Detection: When Language Stops Conditioning the Decision — Current multimodal anomaly detection systems exhibit superficial text guidance, often relying more on visual features than language conditioning.
• Assessing LLM Reliability on Temporally Recent Open-Domain Questions — LLMs achieve high semantic alignment through paraphrasing, not lexical reproduction, challenging traditional evaluation metrics.
• Text-as-Signal: Quantitative Semantic Scoring with Embeddings, Logprobs, and Noise Reduction — Text can be transformed into operational, continuous semantic data points for AI engineering via embeddings and logprob scoring.
• Fully Automated Identification of Lexical Alignment and Preference-Stage Shifts in Large Language Models — Automated metrics can identify and quantify lexical overuse and preference learning shifts in LLMs without manual curation.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.