A Mechanistic Account of Attention Sinks in GPT-2: One Circuit, Broader Implications for Mitigation

2026-04-16 · Source: Machine Learning · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, quick

Summary

Transformers, specifically GPT-2-style models utilizing learned query biases and absolute positional embeddings, frequently display an "attention sink" where the first token receives disproportionately high attention. A study combining structural analysis and causal interventions, validated across natural language, mathematical, and code inputs, reveals this behavior stems from the interaction of a learned query bias, the first-layer MLP transformation of the positional encoding, and specific key projection structure. Each identified component is individually dispensable, meaning architectures lacking one still exhibit sinks, suggesting that attention sinks can emerge via different circuits across various architectures. These findings are crucial for developing mitigation strategies and understanding the underlying reasons for sink emergence.

Key takeaway

For research scientists investigating Transformer model behavior, understanding that attention sinks are not tied to a single architectural component but rather emerge from complex interactions is critical. Your mitigation strategies should therefore target the interplay of learned query biases, positional encoding MLPs, and key projections, rather than isolated elements, to effectively address this robust phenomenon across diverse model designs.

Key insights

Attention sinks in Transformers arise from complex interactions, not single components, and can manifest through diverse circuits.

Principles

• Attention sinks are robust across architectures.
• Multiple circuit paths can lead to attention sinks.

Method

The study combined structural analysis with causal interventions, validated across natural language, mathematical, and code inputs, to identify attention sink mechanisms.

In practice

• Investigate learned query biases.
• Analyze first-layer MLP transformations.
• Examine key projection structures.

Topics

• Attention Sinks
• GPT-2 Models
• Learned Query Bias
• Positional Encoding
• Key Projection

Best for: Research Scientist, AI Scientist, Machine Learning Engineer

Related on AIssential

• A Unifying View of Attention Sinks: Two Algorithms, Two Solutions — Visually similar attention sink patterns in transformers can hide two distinct underlying computational mechanisms.
• [Hands-On] GPT-OSS 바닥부터 구현하기 (3/5)- Self-Attention Mechanism — Implementing Self-Attention with GQA, Attention Sinks, and RoPE significantly optimizes Transformer model efficiency and contextual understanding.
• How Attention Sinks Emerge in Large Language Models: An Interpretability Perspective — Attention sinks on the first token in LLMs emerge early via a non-semantic P0 Sink Circuit.
• The Spike, the Sparse and the Sink: Anatomy of Massive Activations and Attention Sinks — Massive activations and attention sinks in Transformers serve distinct functions, with their co-occurrence driven by pre-norm architecture.
• Is Attention sink without Positional Encoding unavoidable? [D] — Removing Positional Encoding from Transformer attention mechanisms leads to "attention sink" behavior.
• Decoder only Transformer : Building a GPT-2 model prototype to make it understand Natural Language… — Training a GPT-2 prototype on classic novels demonstrates decoder-only Transformer architecture and performance benefits of pre-trained weights.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Machine Learning.