Discovering Lexical Gaps Using Embeddings from Multilingual LLMs

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Expert, medium

Summary

Yoonwon Jung, Aaron S. Cohen, and Ben Bergen propose a data-driven framework for identifying cross-lingual lexical gaps, which are words absent in certain languages and challenge multilingual resource building and machine translation. Published in the Proceedings of the 30th Conference on Computational Natural Language Learning in July 2026, their method utilizes contextualized embeddings from Korean-English bilingual LLMs. They extracted embeddings for Korean-to-English and English-to-Korean translation pairs, generating 4000 distinct embedding spaces across 100 train-test splits by varying LLMs, embedding types, dimensionality, and orthogonal transformations. The research found that gap words exhibited weaker cross-lingual semantic alignment in 94% (Korean-to-English) and 97% (English-to-Korean) of embedding spaces. Logistic classifiers trained on these spaces achieved AUCs of 0.81 (Korean-to-English) and 0.76 (English-to-Korean), successfully retrieving 18/19 Korean and 26/27 English gap words, demonstrating a scalable, language-agnostic, and taxonomy-free approach.

Key takeaway

For NLP Engineers building multilingual systems, this research offers a robust, data-driven alternative to manual lexical gap identification. You can now utilize multilingual LLM embeddings to automatically detect words lacking direct translation equivalents, improving machine translation quality and cross-lingual transfer. Consider integrating this embedding-based classification approach to enhance your lexical resource development and reduce reliance on human judgments or fixed taxonomies.

Key insights

Cross-lingual lexical gaps can be identified by measuring semantic alignment weakness in multilingual LLM embeddings.

Principles

• Lexical gaps show weaker cross-lingual semantic alignment.
• Embedding spaces can be used for gap word classification.
• Data-driven methods offer taxonomy-free gap identification.

Method

Extract contextualized embeddings from bilingual LLMs for translation pairs. Compute semantic similarity between source words and target nearest neighbors. Compare distributions for gap vs. non-gap words to train logistic classifiers.

In practice

• Apply to new language pairs for automated gap discovery.
• Integrate into machine translation systems for improved handling.
• Use to enhance multilingual lexical resource creation.

Topics

• Lexical Gaps
• Multilingual LLMs
• Cross-lingual Embeddings
• Semantic Alignment
• Machine Translation
• Natural Language Processing

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• Fully Automated Identification of Lexical Alignment and Preference-Stage Shifts in Large Language Models — Automated metrics can identify and quantify lexical overuse and preference learning shifts in LLMs without manual curation.
• Token Embeddings : From numbers to meaning — Token embeddings convert arbitrary token IDs into dense, learned vectors that capture semantic relationships through contextual co-occurrence.
• IndicDB -- Benchmarking Multilingual Text-to-SQL Capabilities in Indian Languages — IndicDB reveals a significant "Indic Gap" in LLM Text-to-SQL performance for Indian languages compared to English.
• Align Once, Benefit Multilingually: Enforcing Multilingual Consistency for LLM Safety Alignment — A plug-and-play MLC loss improves multilingual LLM safety alignment using only prompt variants, reducing resource needs.
• Global vs. Local Sentence Embeddings for Brazilian Portuguese: Revisiting Monolingual Models in the Age of Foundation Models — Optimal model choice for PT-BR sentence embeddings balances vocabulary coverage and adaptation flexibility.
• Beyond Surface Forms: A Comprehensive, Mechanism-Oriented Taxonomy of Indirect Linguistic Encoding for LLM-Based Coded Language Detection — A mechanism-oriented taxonomy of indirect linguistic encoding significantly enhances LLM-based coded language detection.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.