Exploring Language-Agnosticity in Function Vectors: A Case Study in Machine Translation

2026-04-21 · Source: Takara TLDR - Daily AI Papers · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Natural Language Processing · Depth: Expert, medium

Summary

A study published on April 21, 2026, investigates the language-agnostic properties of function vectors (FVs) in machine translation, using three decoder-only multilingual large language models (LLMs). Function vectors are task representations derived from model activations during in-context learning. The research demonstrates that translation FVs extracted from an English-to-Target language direction successfully transfer to other unseen target languages, consistently improving the ranking of correct translation tokens. Ablation experiments further confirm that removing these FVs degrades translation performance across languages, while having minimal impact on unrelated tasks. The study also reveals that base-model FVs can transfer effectively to instruction-tuned variants and show partial generalization from word-level to sentence-level translation tasks.

Key takeaway

For research scientists developing multilingual LLMs, understanding that function vectors are language-agnostic means you can potentially reduce training data requirements for new language pairs. You should explore extracting FVs from high-resource languages like English and applying them to improve translation performance in low-resource target languages, thereby enhancing model efficiency and generalization.

Key insights

Function vectors extracted from multilingual LLMs exhibit language-agnostic properties, transferring across diverse target languages for machine translation.

Principles

• FVs improve translation token rank.
• FV removal degrades translation.
• Base-model FVs transfer to instruction-tuned.

Method

FVs are extracted from multilingual LLM activations during in-context learning for English-to-Target translation, then tested for transferability to other target languages and generalization across translation granularities.

In practice

• Use English-trained FVs for new languages.
• Apply FVs to instruction-tuned models.
• Explore FVs for sentence-level translation.

Topics

• Function Vectors
• Language-Agnosticity
• Machine Translation
• Multilingual LLMs
• In-Context Learning

Code references

• ArmelRandy/compositional-translation
• ArmelRandy/ICL-MT

Best for: Research Scientist, AI Scientist, NLP Engineer, Machine Learning Engineer

Related on AIssential

• Exploring Language-Agnosticity in Function Vectors: A Case Study in Machine Translation — Function vectors extracted from multilingual LLMs demonstrate language-agnostic transferability in machine translation tasks.
• mmPISA-bench: Do LLMs Reason Equally Well Across 43 Languages? — LLMs reason effectively across 43 languages, with machine translation not degrading accuracy.
• Disentangling meaning from language in LLM-based machine translation — LLM machine translation functions are disentangled into distinct, specialized attention head sets for language and meaning.
• The Evolving Landscape of LLM Evaluation — Recent AI advancements include top open-weights models, zero-shot MT, and evolving job market trends.
• Large Language Models Do Not Always Need Readable Language — LLMs can effectively process compact, non-human-readable "BabelTele" representations, preserving semantics and reducing context overhead.
• Multilingual Coreference Resolution via Cycle-Consistent Machine Translation — Coreference resolution in low-resource languages can be significantly improved by cycle-consistent machine translation data augmentation.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Takara TLDR - Daily AI Papers.