Controlling Cross-Lingual Answer Distributions in Language Models: Enabling Transfer of Factual Preferences

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Natural Language Processing · Depth: Expert, short

Summary

A study by Ellinger, Manev, and Groh, presented at StereACuLT 2026, investigates controlling cross-lingual answer distributions in multilingual large language models. While prior work focused on reducing inconsistencies, this research explores whether answer distributions from other languages can be expressed under English prompting. The authors constructed a human-annotated factual dataset and a cultural scenarios dataset to evaluate intervention methods. They compared persona prompting, activation steering, and preference-based fine-tuning. Published on pages 35-49, the results demonstrate that answer distributions can be systematically shifted toward those observed in other languages, with persona prompting consistently outperforming the more complex intervention methods in effectiveness and generalization to culturally grounded settings.

Key takeaway

For NLP Engineers or AI Scientists managing multilingual LLM behavior, you should prioritize persona prompting when aiming to transfer factual preferences or cultural nuances across languages. This method consistently outperformed more complex techniques like activation steering and preference-based fine-tuning. Implementing persona prompting can significantly improve your model's consistency and cultural relevance, ensuring outputs align with target language expectations even when prompted in English.

Key insights

Cross-lingual answer distributions in LLMs can be controlled, with persona prompting proving most effective for transferring factual preferences.

Principles

• Multilingual LLMs exhibit systematic output differences.
• Persona prompting effectively shifts LLM answer distributions.
• Simpler intervention methods can outperform complex ones.

Method

The study constructed human-annotated factual and cultural datasets, then compared persona prompting, activation steering, and preference-based fine-tuning to shift LLM answer distributions.

In practice

• Prioritize persona prompting for cross-lingual control.
• Develop human-annotated datasets for cultural scenarios.

Topics

• Multilingual LLMs
• Cross-lingual Transfer
• Persona Prompting
• Answer Distributions
• Cultural Bias
• Factual Preferences

Best for: Research Scientist, AI Engineer, Machine Learning Engineer, AI Scientist, NLP Engineer, Prompt Engineer

Related on AIssential

• English is Not All You Need: Systematically Exploring the Role of Multilinguality in LLM Post-Training — Multilingual post-training significantly improves LLM performance across languages and tasks, outperforming English-only approaches.
• The System Prompting Guide for The Business Engineer — Effective LLM prompting requires a systems thinking mental model, viewing prompts as interventions in a probabilistic distribution.
• Creating Multilingual Mental Health Dialogue Datasets: Limits of Persona-Based Localization via Nationality and Language — Simple nationality and language parameter changes in English-centric personas fail to preserve clinical consistency in multilingual mental health dialogues.
• LLMs Infer Cultural Context but Fail to Apply It When Responding — LLMs infer cultural context but often fail to apply it in responses without explicit sequential prompting, revealing cultural biases.
• Extending the Contact Hypothesis: Cross-Linguistic Evaluation of Religion and Nationality Bias When Prompting LLMs in German and Icelandic — LLMs exhibit religion and nationality biases, requiring cross-linguistic evaluation for fairness.
• Cross-Lingual Consensus: Aligning Multilingual Cultural Knowledge via Multilingual Self-Consistency — LLMs' latent cultural knowledge in local languages can be surfaced and transferred to improve English query performance and reduce Western bias.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.