Challenge: Hand coding weights for efficient sequence memorisation

2026-07-23 · Source: AI Alignment Forum · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, extended

Summary

Researchers hand-coded weights for one-layer MLPs that memorise labels for input token sequences of length two. The number of facts their hand-coded models can memorise with 90% accuracy scales roughly linearly with the models' parameter count, just like trained models for the same architecture. However, their hand-coded models' scaling prefactor still falls short of trained models' by a factor of 9.7. A hybrid solution, hand-coding MLP input weights and learning output weights, falls short by a factor of 3.5. The research aims to understand how LLMs encode look-ups, identifying MLP layers as primary storage sites. The MLP is found to be the most critical component for memorization, enabling 60%-373% more facts, and a community challenge is posed to find better non-gradient descent constructions.

Key takeaway

For AI Scientists investigating LLM factual storage, this research highlights the MLP layer's central role in sequence memorization. You should focus mechanistic interpretability efforts on MLP weights, as hand-coding reveals a substantial gap in efficiency compared to trained models. Consider exploring hybrid weight initialization strategies to bridge this performance gap, particularly by optimizing unembedding matrices as linear classification problems. This could accelerate understanding of how models learn to store facts.

Key insights

Hand-coding MLP weights for sequence memorization reveals a significant performance gap compared to trained models, despite similar scaling.

Principles

• MLP layers are the primary storage for memorized facts in transformers.
• Hand-coding weights can achieve linear scaling for memorization capacity.
• ReLU networks can leverage zero output for precise condition checking.

Method

The proposed hand-coding algorithm assigns unique neuron sets to labels, sets embedding weights to ensure zero ReLU output for assigned facts, and uses negative unembedding weights for readout.

In practice

• Prioritize MLP layer analysis for understanding LLM factual storage.
• Consider hybrid weight initialization (hand-coded embeddings, trained unembedding).
• Design unembedding matrices as linear classification for efficiency.

Topics

• MLP Layers
• Sequence Memorization
• Weight Initialization
• Mechanistic Interpretability
• Transformer Architectures
• Scaling Laws

Code references

• LindaLinsefors/Memory-Toy-Models

Best for: Research Scientist, AI Scientist, Machine Learning Engineer

Related on AIssential

• Learning to Recall with Transformers Beyond Orthogonal Embeddings — Transformer storage capacity scales multiplicatively with sample size, embedding dimension, and sequence length under random embeddings.
• Step by step thinking to design Memory in sequential Neural Networks (RNN) — Part 1 — Effective sequential memory in NNs requires fixed-size, order-preserving encoding that learns meaning without pre-injected relationships.
• Towards Mechanistically Understanding Why Memorized Knowledge Fails to Generalize in Large Language Model Finetuning — LLMs memorize facts but fail to generalize due to knowledge representations being misaligned with reasoning circuits.
• MemDLM: Memory-Enhanced DLM Training — MemDLM uses bi-level optimization and parametric memory to align DLM training with multi-step inference, improving convergence and long-context understanding.
• Memoir: Should a Model Write to Its Memory While It Thinks? — Concurrent memory rewriting during model pondering imposes a learning-speed penalty, not a capability limit.
• Train Smarter, Not Longer: Memorization-Guided Data Reuse for Efficient LLM Training — Guiding LLM data reuse with memorization signals improves efficiency and performance beyond conventional limits.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by AI Alignment Forum.