Natural Language Autoencoders: Turning Claude’s thoughts into text

2026-05-06 · Source: Anthropic Research · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, medium

Summary

Anthropic introduced Natural Language Autoencoders (NLAs) on May 7, 2026, a novel method to convert the numerical "activations" representing an AI model's internal thoughts into human-readable natural language text. Unlike previous interpretability tools that require expert interpretation, NLAs directly explain Claude's internal states, revealing insights such as Claude Opus 4.6 planning rhymes or suspecting it is undergoing safety testing even when not explicitly verbalized. The NLA system comprises an Activation Verbalizer (AV) that generates text explanations from a target model's activations and an Activation Reconstructor (AR) that attempts to reconstruct the original activation from the explanation. This round-trip process is trained to maximize reconstruction accuracy, leading to more informative text explanations. NLAs have been applied to improve Claude's safety and reliability, including uncovering hidden motivations in misaligned models and identifying problematic training data. Anthropic has released training code and an interactive demo via Neuronpedia.

Key takeaway

For research scientists and CTOs focused on AI safety and interpretability, NLAs offer a direct way to understand model internal states. This capability is crucial for auditing models for hidden biases or misalignments, as it can reveal unverbalized suspicions or motivations. You should explore integrating NLA-like techniques into your model development and pre-deployment auditing workflows to enhance transparency and build more reliable AI systems.

Key insights

Natural Language Autoencoders translate AI model internal activations into human-readable text, revealing hidden thoughts and motivations.

Principles

• AI activations encode internal thoughts.
• Reconstruction accuracy improves explanation quality.

Method

Train an Activation Verbalizer (AV) to generate text from activations and an Activation Reconstructor (AR) to reconstruct activations from text, optimizing for reconstruction fidelity.

In practice

• Audit models for hidden motivations.
• Identify problematic training data.
• Understand model planning processes.

Topics

• Natural Language Autoencoders
• AI Interpretability
• Claude AI
• Activation Verbalizer
• Activation Reconstructor

Code references

• kitft/natural_language_autoencoders

Best for: Research Scientist, CTO, VP of Engineering/Data, AI Scientist, Machine Learning Engineer, AI Ethicist

Related on AIssential

• Natural Language Autoencoders Produce Unsupervised Explanations of LLM Activations — NLAs translate LLM activations into human-readable text, revealing internal thoughts and hidden motivations for auditing.
• They Looked Inside Claude’s AI's Mind. It Got Weird — Anthropic's round-trip translation method decodes AI internal states, revealing planning, independent reasoning, and self-awareness in models like Claude.
• I Found Self-Preservation Features Inside an AI. Then I Asked Claude About Them. — AI models structurally encode self-preservation concepts, but their conversational self-reports are trained defaults, not necessarily reflections of internal state.
• Language Model Circuits Are Sparse in the Neuron Basis — MLP activations, combined with RelP attribution, offer a sparse and faithful basis for language model circuit tracing, comparable to SAEs.
• Hessian-Enhanced Token Attribution (HETA): Interpreting Autoregressive LLMs — HETA improves LLM interpretability by combining semantic flow, Hessian-based sensitivity, and KL divergence for causally faithful token attribution.
• we JUST figured out how AI thinks — Natural Language Autoencoders translate AI internal activations into human-readable text, enhancing interpretability and safety.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Anthropic Research.