Exploring Language-Agnosticity in Function Vectors: A Case Study in Machine Translation

2026-04-21 · Source: Computation and Language · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, quick

Summary

A recent study investigates the language-agnostic properties of function vectors (FVs) within multilingual large language models (LLMs), specifically in the context of machine translation. Function vectors are task representations derived from model activations during in-context learning. The research, conducted across three decoder-only multilingual LLMs, demonstrates that translation FVs extracted from an English-to-Target language direction successfully transfer to other target languages. This transfer consistently enhances the ranking of correct translation tokens across multiple previously unseen languages. Ablation studies further confirm that removing these FVs degrades translation performance across languages, while having minimal effect on unrelated tasks. The findings also indicate that base-model FVs can transfer to instruction-tuned variants and exhibit partial generalization from word-level to sentence-level translation.

Key takeaway

For research scientists developing multilingual LLMs, understanding function vector transferability is crucial. You should explore leveraging language-agnostic FVs to improve translation quality and efficiency across diverse target languages, potentially reducing the need for extensive language-specific training data. Consider testing base-model FVs on instruction-tuned variants to streamline model development and enhance generalization.

Key insights

Function vectors extracted from multilingual LLMs demonstrate language-agnostic transferability in machine translation tasks.

Principles

• FVs improve translation token rank.
• FV removal degrades translation.
• Base FVs transfer to instruction-tuned models.

Method

FVs are extracted from multilingual LLM activations during in-context learning for English-to-Target translation, then applied to other target languages to assess transferability.

In practice

• Apply English-trained FVs to new languages.
• Use FVs to enhance translation accuracy.
• Explore FV transfer across model variants.

Topics

• Function Vectors
• Language-Agnosticity
• Machine Translation
• Multilingual LLMs
• In-Context Learning

Best for: Research Scientist, AI Scientist, NLP Engineer

Related on AIssential

• Exploring Language-Agnosticity in Function Vectors: A Case Study in Machine Translation — Function vectors extracted from multilingual LLMs exhibit language-agnostic properties, transferring across diverse target languages for machine translation.
• Disentangling meaning from language in LLM-based machine translation — LLM machine translation functions are disentangled into distinct, specialized attention head sets for language and meaning.
• The Evolving Landscape of LLM Evaluation — Recent AI advancements include top open-weights models, zero-shot MT, and evolving job market trends.
• Translate-R1: Cost-Aware Translation Tool Use via Reinforcement Learning — Reinforcement learning enables LLMs to intelligently decide when to use translation tools, optimizing cost and performance.
• mmPISA-bench: Do LLMs Reason Equally Well Across 43 Languages? — LLMs reason effectively across 43 languages, with machine translation not degrading accuracy.
• Mechanistic Insights into Functional Sparsity in Multimodal LLMs via CoRe Heads — MLLMs exhibit functional sparsity via Context-aware Retrieval (CoRe) heads, crucial for efficient cross-modal information extraction.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computation and Language.