DySem: Uncovering Dynamic Semantic Components via Multilingual Consensus for Calculating Semantic Textual Similarity

2026-05-28 · Source: Computation and Language · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Natural Language Processing · Depth: Expert, quick

Summary

DySem is a novel, training-free framework designed to improve semantic textual similarity (STS) calculation by addressing limitations in current large language model (LLM)-based methods. Existing approaches often rely on extracting last-layer hidden states, which encode general knowledge rather than purely semantic information and feature large, redundant dimensions. DySem investigates more semantic-related internal components of LLMs through multilingual consensus. It shifts from static representation spaces to dynamic, sample-specific semantic dimensions by constructing a text-dependent joint semantic set and computing similarity over this shared dimensional subset. Extensive experiments across various LLMs demonstrate that DySem consistently outperforms recent baselines while maintaining lower dimensions for similarity calculation. The code is publicly available at https://github.com/szu-tera/DySem.

Key takeaway

For NLP Engineers focused on optimizing semantic textual similarity (STS) tasks, consider integrating DySem to enhance performance and efficiency. This training-free framework allows you to move beyond static, high-dimensional LLM representations, achieving more accurate STS calculations with significantly lower dimensions. You should explore DySem's multilingual consensus approach to uncover specific semantic components within your LLMs, potentially reducing computational overhead while improving result quality.

Key insights

DySem uses multilingual consensus to find dynamic, sample-specific semantic dimensions for improved STS in LLMs.

Principles

• LLM last hidden layers encode general, not just semantic, knowledge.
• Static, high-dimensional representations introduce STS redundancy.
• Multilingual consensus can uncover semantic-specific LLM components.

Method

DySem identifies semantic-related LLM internal components via multilingual consensus, then constructs text-dependent joint semantic sets for dynamic, sample-specific similarity calculation.

In practice

• Apply DySem to improve STS with existing LLMs.
• Reduce dimensionality for STS calculations.
• Explore internal LLM components beyond last layer.

Topics

• Semantic Textual Similarity
• Large Language Models
• DySem Framework
• Multilingual Consensus
• Dynamic Embeddings
• NLP Inference Optimization

Code references

• szu-tera/DySem

Best for: AI Engineer, Research Scientist, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• Align Once, Benefit Multilingually: Enforcing Multilingual Consistency for LLM Safety Alignment — A plug-and-play MLC loss improves multilingual LLM safety alignment using only prompt variants, reducing resource needs.
• Building Context-Aware Search in Python with LLM Embeddings + Metadata — Context-aware semantic search combines embedding-based similarity with metadata filtering for relevant results respecting contextual constraints.
• Hybrid Human-LLM Corpus Construction and LLM Evaluation for the Caused-Motion Construction — LLMs struggle with complex linguistic constructions like CMC, necessitating novel corpus creation methods.
• IndicDB -- Benchmarking Multilingual Text-to-SQL Capabilities in Indian Languages — IndicDB reveals a significant "Indic Gap" in LLM Text-to-SQL performance for Indian languages compared to English.
• Ancient Greek to Modern Greek Machine Translation: A Novel Benchmark and Fine-Tuning Experiments on LLMs and NMT Models — A new AG-MG parallel corpus and benchmark significantly advance Ancient Greek to Modern Greek machine translation.
• Comparing LLM and Fine-Tuned Model Performance on NVDRS Circumstance Extraction with Varying Prompt Complexity — LLMs, guided by a context-aware "Complexity Score" for prompt selection, significantly outperform fine-tuned models on inferentially complex, low-prevalence data extraction.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computation and Language.