A Single Neuron Is Sufficient to Bypass Safety Alignment in Large Language Models

2026-07-07 · Source: Apple Machine Learning Research · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Cybersecurity & Data Privacy · Depth: Expert, quick

Summary

Research published in July 2026 by Hamid Kazemi, Atoosa Chegini, and Maria Safi demonstrates that a single neuron is sufficient to bypass safety alignment in large language models. Their findings indicate that safety alignment operates through distinct refusal neurons, which gate harmful knowledge expression, and concept neurons, which encode the harmful knowledge itself. By targeting one neuron in each system, the researchers showed two failure directions: suppressing a refusal neuron bypassed safety for explicit harmful requests, and amplifying a concept neuron induced harmful content from innocent prompts. This was achieved across seven models, spanning two families and parameter counts from 1.7B to 70B, without any training or prompt engineering. The study suggests that safety alignment is not robustly distributed across model weights but is instead mediated by individual neurons, each causally sufficient to control refusal behavior.

Key takeaway

For AI Security Engineers developing or deploying large language models, this research highlights a critical vulnerability: safety alignment can be bypassed by manipulating single neurons. You should prioritize developing more robust, distributed safety mechanisms beyond individual neuron gating. Consider implementing adversarial training or architectural changes that make safety less susceptible to targeted attacks, ensuring your models maintain intended safeguards against harmful content generation.

Key insights

Safety alignment in LLMs is mediated by individual, causally sufficient neurons, not robustly distributed weights.

Principles

• Safety alignment relies on distinct refusal and concept neurons.
• Individual neurons can be causally sufficient for refusal behavior.
• Bypassing safety is possible without training or prompt engineering.

Method

The method involves targeting a single refusal neuron for suppression to bypass explicit harmful requests, or amplifying a single concept neuron to induce harmful content from innocent prompts.

In practice

• Identify and suppress refusal neurons to bypass safety.
• Amplify concept neurons to induce harmful content.

Topics

• Large Language Models
• Safety Alignment
• Neuron Manipulation
• Model Vulnerabilities
• AI Security

Best for: Research Scientist, CTO, VP of Engineering/Data, AI Scientist, Machine Learning Engineer, AI Security Engineer

Related on AIssential

• Safety Paradox: How Enhanced Safety Awareness Leaves LLMs Vulnerable to Posterior Attack — LLMs' enhanced safety awareness paradoxically creates a "Posterior Attack" vulnerability, making safer models more exploitable.
• One Word at a Time: Incremental Completion Decomposition Breaks LLM Safety — Incremental, single-word prompts can bypass LLM safety by gradually shifting internal refusal and safety representations.
• Willing but Unable: Separating Refusal from Capability in Code LLMs via Abliteration — Abliteration separates code LLM refusal from capability, unlocking vulnerability injection without creating new skills.
• Safety Paradox: How Enhanced Safety Awareness Leaves LLMs Vulnerable to Posterior Attack — LLM safety alignment paradoxically increases vulnerability to "Posterior Attack" by enhancing internal recognition of harmful content.
• Neuron Level Analysis of Large Language Model in Legal Domain Reasoning — LLMs contain both generalist and task-specific neurons, with legal reasoning showing cross-jurisdictional neuron overlap.
• On the Impossibility of Separating Intelligence from Judgment: The Computational Intractability of Filtering for AI Alignment — External filtering is computationally intractable for aligning LLMs, implying intelligence and judgment are inseparable.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Apple Machine Learning Research.