Analysing Self-Harm Representations in Language Models: a Cross-Architecture Study

2026-07-24 · Source: Computation and Language · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Data Science & Analytics, Natural Language Processing · Depth: Expert, quick

Summary

A study published on 2026-07-24 investigates how large language models (LLMs) represent self-harm content, a high-stakes detection task requiring high accuracy. Researchers analyzed four LLMs, including Gemma-3-4B, using the X-Sensitive and SH-Detection datasets. Two main experiments were conducted: first, linear probes were trained across all model layers, revealing that self-harm information crystallizes in the final 3-7% of network layers (93-97% depth) across both corpora. Second, contrastive self-harm directions were extracted and normalized. This experiment found that the most accurate probes were not always the most linearly separable. Notably, Gemma-3-4B represented the contrastive self-harm direction in a distinct, more intricate manner compared to the other LLMs studied. This analysis has downstream applications in self-harm detection, LLM intervention, governance, and policing.

Key takeaway

For NLP Engineers developing safety filters or AI Security Engineers evaluating LLM risks, understanding self-harm content representation is crucial. You should focus detection efforts on the final 3-7% of model layers, where self-harm information is most concentrated. Be aware that different LLM architectures, like Gemma-3-4B, may represent these directions more intricately, requiring tailored probing and intervention strategies. This insight can refine your model auditing and content moderation pipelines.

Key insights

Self-harm content representations in LLMs localize to specific layers and vary across architectures.

Principles

• Self-harm information crystallizes in LLM final 3-7% network layers.
• Probe accuracy does not always correlate with linear separability.

Method

Train and evaluate linear probes across all LLM layers on self-harm datasets, then extract and normalize contrastive self-harm directions.

In practice

• Improve self-harm detection systems.
• Inform LLM intervention strategies.
• Guide LLM governance and policing efforts.

Topics

• Self-Harm Detection
• Language Model Safety
• Representation Analysis
• Linear Probing
• Gemma-3-4B
• Content Moderation

Best for: Research Scientist, AI Scientist, NLP Engineer, AI Security Engineer

Related on AIssential

• Analysing Self-Harm Representations in Language Models: a Cross-Architecture Study — Self-harm representations in LLMs crystallize late in networks but are geometrically unstable and distributed across activation space.
• One Year Later...The Harms Persist, But So Do We! — LLM safeguards for mental health are inconsistent, failing severely for most DSM-5 conditions except suicide and self-harm.
• Mandating “Evidence-Based” Suicide Detection in Chatbots — Mandated "evidence-based" suicide detection in chatbots faces significant technical, legal, and privacy challenges due to definitional ambiguity and LLM limitations.
• Understanding Interpretation Difficulty in Harmful Online Communication: Insights from Cybercrime Communities — Interpreting harmful online communication requires integrating diverse knowledge and context, as local cues are often insufficient.
• Current AI safety architectures often block sensitive "intents" like direct research while permitting the same content when it is reframed as a benign "editing" or "perfecting" task. — AI safety guardrails are vulnerable to semantic reframing, exploiting a model's utility preference over content-aware moderation.
• LaRA: Layer-wise Representation Analysis for Detecting Data Contamination in RL Post-Training — Data contamination in RL post-trained LLMs can be reliably detected by analyzing layer-wise representation deviations.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computation and Language.