Soft Token Alignment for Cross-Lingual Reasoning

2026-06-25 · Source: Computation and Language · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, quick

Summary

Multilingual large language models often exhibit inconsistent reasoning across languages for semantically equivalent prompts, a problem stemming from language-specific lexical choices that cause reasoning paths to diverge. To address this, a new auxiliary objective called SOLAR is proposed for supervised fine-tuning. SOLAR aligns "soft-token representations" across languages, using English as a pivot. Soft tokens are continuous, probability-weighted mixtures of vocabulary embeddings, designed to aggregate information from semantically related tokens. By aligning non-English soft-token summaries to their English counterparts in a shared embedding space, SOLAR significantly improves accuracy. It achieves up to +17.7 points over base models and +3.8 points over standard supervised fine-tuning on four multilingual reasoning benchmarks, with the most substantial gains observed in low-resource languages. This approach also enhances final-layer cross-lingual similarity and reduces language-cluster separability, indicating better preservation of shared semantic structure.

Key takeaway

For Machine Learning Engineers developing multilingual LLMs, consider integrating soft-token alignment techniques like SOLAR. This approach can significantly improve reasoning consistency and accuracy, particularly for low-resource languages, by preserving shared semantic structure across languages. Your models will exhibit stronger cross-lingual similarity and reduced language-cluster separability, leading to more reliable performance on diverse linguistic inputs. Explore fine-tuning with auxiliary objectives that explicitly align continuous representations.

Key insights

SOLAR aligns soft-token representations across languages using English as a pivot to enhance multilingual LLM reasoning consistency and accuracy.

Principles

• Multilingual LLM reasoning paths diverge from language-specific lexical choices.
• Intermediate representations are more language-agnostic than discrete outputs.
• Soft tokens aggregate semantic information across languages effectively.

Method

SOLAR, an auxiliary objective for supervised fine-tuning, aligns non-English soft-token summaries to their English counterparts in a shared embedding space, using English as a pivot.

In practice

• Improve multilingual LLM accuracy, especially for low-resource languages.
• Enhance final-layer cross-lingual similarity.
• Reduce language-cluster separability in embeddings.

Topics

• Multilingual LLMs
• Soft Token Alignment
• Cross-Lingual Reasoning
• Supervised Fine-tuning
• Low-Resource Languages
• Semantic Structure Preservation

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, NLP Engineer

Related on AIssential

• Align Once, Benefit Multilingually: Enforcing Multilingual Consistency for LLM Safety Alignment — A plug-and-play MLC loss improves multilingual LLM safety alignment using only prompt variants, reducing resource needs.
• LLM Parameters for Math Across Languages: Shared or Separate? — LLM mathematical reasoning involves partially shared and partially language-specific parameters, with English dominating parameter sets.
• Linguistic Issues in Computational Linguistics for Non-English Languages — English-centric LLM design creates systematic linguistic weaknesses for non-English languages across multiple levels.
• Code-Switching Reveals Language Anchoring in Multilingual LLMs — Grammatical frame dictates multilingual LLM internal language anchoring, impacting code-switched performance.
• G-IdiomAlign: A Gloss-Pivoted Benchmark for Cross-Lingual Idiom Alignment — Cross-lingual idiom alignment challenges LLMs due to non-compositionality, with glosses offering semantic grounding but literal bias remaining dominant.
• Beyond Entropy: Learning from Token-Level Distributional Deviations for LLM Reasoning — ICT stabilizes LLM reasoning by focusing on token-level distributional deviations, preventing entropy collapse and explosion.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computation and Language.