Localizing Anchoring Pathways in Language Models

2026-06-11 · Source: Artificial Intelligence · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, quick

Summary

A study published on 2026-06-11 investigates how irrelevant numerical information in prompts creates anchoring effects in language model judgments, particularly in numerical reasoning tasks. Researchers employed a controlled multiple-choice setup and defined a logit-difference metric to track behavioral anchoring. Using attribution-based circuit localization on 7B--8B Qwen and Llama base and instruction-tuned models, the analysis revealed that edge-level methods more faithfully recover anchor-sensitive signals than node-level methods. The findings indicate strong transfer of low- and high-anchor circuits within a single model, suggesting shared pathway structures. However, transfer across base and instruction-tuned variants was sparse, highlighting that post-training significantly alters which internal pathways are most critical for these decisions. This work provides a mechanistic account of how anchoring-related decision signals are carried within language models.

Key takeaway

For AI Scientists and Machine Learning Engineers focused on model robustness and fairness, understanding the mechanistic basis of biases like anchoring is crucial. This research demonstrates that anchoring effects are localized to specific internal pathways within models, with post-training significantly influencing these critical decision circuits. You should prioritize investigating how fine-tuning or instruction-tuning modifies internal decision-making to build more reliable and less susceptible language models.

Key insights

Irrelevant numerical anchors in prompts shift LM judgments via specific, localizable internal pathways.

Principles

• Anchoring effects are mechanistically localizable within LMs.
• Edge-level attribution excels in circuit localization.
• Post-training alters critical decision pathways.

Method

A logit-difference metric tracks behavioral anchoring. Attribution-based circuit localization on multiple-choice setups identifies anchor-sensitive signal pathways.

In practice

• Use edge-level attribution for LM circuit analysis.
• Consider post-training impact on model decision pathways.

Topics

• Language Models
• Anchoring Effects
• Circuit Localization
• Attribution Methods
• Qwen
• Llama
• Instruction Tuning

Best for: Research Scientist, AI Scientist, Machine Learning Engineer

Related on AIssential

• Localizing Anchoring Pathways in Language Models — Anchoring bias in LLMs is localized to sparse, shared internal pathways, but these pathways shift with instruction tuning.
• The Value Axis: Language Models Encode Whether They're on the Right Track — Language models internally track their trajectory's likelihood of goal success via a linearly encoded "value" axis.
• Unintended Effects of Geographic Conditioning in Large Language Models — LLMs exhibit "location leakage" and structural conditioning, generating regional biases even with neutral prompts.
• Weight Patching: Toward Source-Level Mechanistic Localization in LLMs — Weight Patching localizes LLM behaviors to specific parameters by transferring weights between specialized and base models.
• MENTIS: What Belief Changes Under Alignment? Measuring Multi-Scale Latent Torsion in Language Models — Alignment selectively reorganizes LLM internal geometry, with measurable torsion shifts concentrated in specific layers and concepts.
• Pattern Selectivity is Not Task-Causal Structure: A Cross-Architecture Mechanistic Study of Composed-Task Circuits in 1B-Class Language Models — Specific mechanistic circuits for composed tasks do not transfer across 1B-class language models.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Artificial Intelligence.