PII Jailbreaking in LLMs via Activation Steering Reveals Personal Information Leakage

2026-06-25 · Source: Paper Index on ACL Anthology · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Cybersecurity & Data Privacy · Depth: Expert, quick

Summary

A study investigates privacy jailbreaking in large language models (LLMs) using "activation steering" to circumvent alignment mechanisms and extract personal identifiable information (PII). Researchers identified attention heads predictive of refusal behavior for private attributes, like public figures' sexual orientation. They used lightweight linear probes trained with a privacy evaluator. By applying steering to a selected subset of these heads, guided by probe outputs, LLMs were induced to provide positive responses to privacy-sensitive queries. Empirical results showed steered responses frequently revealed the target attribute and additional PII, including life events, relationships, and biographical details. Evaluations across three LLMs demonstrated disclosure rates of at least 80%, with several responses containing real personal information. This controlled study highlights a concrete privacy risk: memorized PII can be extracted via targeted activation-level interventions, without computationally intensive adversarial prompting.

Key takeaway

For AI Security Engineers evaluating LLM privacy, you must consider activation steering as a potent PII leakage vector, not just adversarial prompting. Your current alignment mechanisms may be insufficient against internal manipulation that can achieve over 80% disclosure rates of memorized personal data. Prioritize developing and implementing defenses that monitor and control internal model activations to prevent unauthorized PII extraction.

Key insights

Activation steering can jailbreak LLMs to leak memorized PII, posing a significant privacy risk without adversarial prompting.

Principles

• Internal activation manipulation bypasses LLM alignment.
• PII memorized during pre-training is extractable.
• Targeted interventions can induce specific model behaviors.

Method

Identify refusal-predictive attention heads via linear probes, then apply steering to a subset of these heads, guided by probe outputs, to induce PII disclosure.

In practice

• Assess LLM privacy risks beyond adversarial prompts.
• Develop defenses against activation-level PII extraction.
• Enhance alignment mechanisms against internal steering.

Topics

• Large Language Models
• Activation Steering
• PII Jailbreaking
• Privacy Risk
• Information Leakage
• Attention Heads

Best for: CTO, VP of Engineering/Data, Director of AI/ML, AI Scientist, AI Security Engineer

Related on AIssential

• LLM Self-Recognition: Steering and Retrieving Activation Signatures — LLMs can reliably self-recognize their outputs, a capability enhanced by sparse internal activation steering for robust attribution.
• The Day I Realized Our LLM Was Quietly Logging Patient SSNs — Data protection for LLMs is an architectural problem requiring a robust redaction gate, not just instruction.
• Persona Attack: Incremental Memory Injection Jailbreak Attack against Large Language Models — Persona Attack exploits LLM conversational memory to bypass safety alignment via incremental instruction injection.
• It Takes Two: Complementary Self-Distillation for Contextual Integrity in LLMs — SelfCI decouples privacy and utility in LLMs using complementary self-distillation to achieve Contextual Integrity, outperforming RL baselines.
• Actionable Activation Directions for Detecting and Mitigating Emergent Misalignment Across Language Model Families — Emergent misalignment in fine-tuned LMs corresponds to causally actionable activation-space directions, enabling within-model mitigation.
• Adversarial Robustness of Activation Steering in Large Language Models — Activation steering in LLMs exhibits structural brittleness against adversarial text perturbations, significantly degrading control and making it unsuitable for robust deployment.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Paper Index on ACL Anthology.