Reading and Steering Representations of Materials-Science Mechanisms in an Open-Weight Language Model

2026-07-16 · Source: cs.CL updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Emerging Technologies & Innovation, Scientific AI Interpretability · Depth: Expert, extended

Summary

The paper "Reading and Steering Representations of Materials-Science Mechanisms in an Open-Weight Language Model" by Markus J. Buehler investigates how the open-weight google/gemma-4-E4B-it model represents and uses materials science mechanisms. The study identifies three forms of information: concepts readable in individual hidden states, constitutive orientation in controlled state transformations, and causal control over engineering answers by selected internal representations. Using matched direct and Jacobian vocabulary readouts, a 60-law counterfactual benchmark, and causal interventions, the research found that Jacobian lenses reproduced concept ranks in 50 materials descriptions, identifying 9 of 10 mechanism families. While apparent physical organization was observed in hidden-state neighborhoods, a graph audit showed it was equally explained by numerical comparison. Crucially, state transformations correctly oriented 39 of 40 directional laws, and bidirectional interventions shifted answer probabilities towards physically appropriate outcomes across 12 matched cases. The study concludes that physical relationships are more visible in controlled state changes than in absolute states alone.

Key takeaway

For AI Scientists and ML Engineers evaluating LLM reliability in scientific domains, recognize that correct answers don't guarantee true physical understanding. Focus on analyzing internal state transformations, not just static representations, to verify constitutive law orientation. Implement causal interventions to confirm if internal directions predictably steer scientific decisions. This approach helps diagnose failures and build more robust, scientifically grounded AI systems.

Key insights

LLMs represent scientific mechanisms more reliably through state transformations than static states.

Principles

• LLM internal states can encode scientific concepts.
• Constitutive orientation is visible in state changes.
• Causal intervention validates internal representations.

Method

Combines direct/Jacobian vocabulary readouts, option-free state geometry analysis, a 60-law counterfactual benchmark, and causal interventions to probe LLM internal states.

In practice

• Use Jacobian lenses for concept readability.
• Employ state transformations for physical law orientation.
• Apply causal interventions to steer scientific decisions.

Topics

• Mechanistic Interpretability
• Large Language Models
• Materials Science
• Gemma Model
• Jacobian Lens
• Causal Intervention

Code references

• lamm-mit/Substrates

Best for: AI Scientist, Machine Learning Engineer, Research Scientist

Related on AIssential

• Reading and Steering Representations of Materials-Science Mechanisms in an Open-Weight Language Model — Large language models can represent and causally utilize governing materials science physics in distinct, measurable internal forms.
• Breaking the Chain: A Causal Analysis of LLM Faithfulness to Intermediate Structures — LLMs' apparent faithfulness to intermediate reasoning structures is fragile; they often fail to update predictions after causal intervention.
• A family of large language models for materials research with insights into model adaptability in continued pretraining — Domain-adapted LLMs significantly enhance materials research, but overtraining can hinder further specialization.
• Mechanistic Interpretability: Peeking Inside an LLM — Mechanistic interpretability dissects LLM internal states to understand and control their emergent behaviors and knowledge.
• What LLM Forecasters Know but Don't Say: Probing Internal Representations for Calibration and Faithfulness — Probing LLM internal representations reveals true forecast confidence and evidence sensitivity better than Chain-of-Thought.
• Think like a Scientist: Physics-guided LLM Agent for Equation Discovery — KeplerAgent improves equation discovery by integrating scientific reasoning and physics-guided priors into LLM agents.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CL updates on arXiv.org.