Cross-Lingual Consensus: Aligning Multilingual Cultural Knowledge via Multilingual Self-Consistency

2026-05-22 · Source: cs.CL updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics · Depth: Expert, extended

Summary

A novel self-supervised framework is proposed to align multilingual cultural knowledge in Large Language Models (LLMs), addressing performance disparities and Western-centric bias when models are prompted in English. This method, called "Cross-Lingual Consensus," leverages multilingual self-consistency to identify the most reliable cultural responses across different languages. It then employs a self-critique mechanism to transfer this robust knowledge to languages where the model performs weaker. Evaluations on the BLEnD benchmark demonstrated that this approach significantly improved cultural alignment, boosting performance on English queries by an average of 5.03%. The framework generates its own training data, adapting CulFiT by omitting the Direct Preference Optimization phase and using Bilingual Question Generation from CANDLE and CultureAtlas datasets, followed by Self-Supervised Ground Truth Generation based on response consistency using Qwen3-Embedding-0.6B embeddings. Training involves LoRA with rank r=16 on 8 H200 GPUs for 1,000 steps.

Key takeaway

For NLP Engineers developing multilingual LLMs, you should consider implementing self-supervised cross-lingual consistency methods to mitigate Western-centric bias and improve cultural alignment. This approach allows you to surface latent cultural knowledge from local language representations, enhancing model reliability in diverse global contexts without costly human annotation. Integrate self-critique mechanisms to effectively transfer robust cultural insights to weaker language performance areas, ensuring more equitable and consistent LLM outputs.

Key insights

LLMs can self-align cultural knowledge across languages by identifying consistent responses and transferring them to weaker language contexts.

Principles

• LLMs possess latent cultural knowledge in local languages.
• Cross-lingual consistency can identify reliable cultural responses.
• Self-supervision can improve cultural alignment without human annotation.

Method

The framework generates bilingual question pairs, samples N responses per language, calculates intra-language consistency using Qwen3-Embedding-0.6B embeddings, and translates the most consistent stronger-language answer as ground truth for weaker languages.

In practice

• Use multilingual self-consistency for data synthesis.
• Apply self-critique to transfer knowledge to weaker languages.
• Fine-tune LLMs with LoRA for cultural alignment.

Topics

• Multilingual LLMs
• Cultural Bias Mitigation
• Self-Consistency Framework
• Cross-Lingual Knowledge Transfer
• Self-Supervised Alignment
• BLEnD Benchmark

Best for: Research Scientist, AI Scientist, NLP Engineer, AI Ethicist

Related on AIssential

• Cross-Lingual Consensus: Aligning Multilingual Cultural Knowledge via Multilingual Self-Consistency — LLMs' latent cultural knowledge in local languages can be surfaced and transferred to improve English query performance and reduce Western bias.
• AI’s fluency in other languages hides a Western worldview that can mislead users − a scholar of Indonesian society explains — LLMs exhibit "epistemological persistence," retaining a Western worldview despite multilingual fluency due to English-centric training data.
• CulturALL: Benchmarking Multilingual and Multicultural Competence of LLMs on Grounded Tasks — CulturALL benchmarks LLMs' multilingual and multicultural reasoning in grounded, real-world tasks.
• Rethinking Cross-lingual Gaps from a Statistical Viewpoint — LLM cross-lingual gaps stem from response variance, not knowledge barriers, and are reducible via variance control.
• Research on LLM alignment as latent discourse-level regimes vs. token-level filtering? — LLM alignment may stem from global latent space geometry and discourse-level regimes, not just token-level filtering.
• CuMA: Aligning LLMs with Sparse Cultural Values via Demographic-Aware Mixture of Adapters — CuMA uses demographic-aware routing and specialized adapters to prevent LLMs from collapsing diverse cultural values into a generic average.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CL updates on arXiv.org.