Enhancing Safety of Large Language Models via Embedding Space Separation

2026-03-24 · Source: cs.CL updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Cybersecurity & Data Privacy · Depth: Expert, extended

Summary

A novel fine-tuning approach, Embedding Space Separation (ES2), significantly enhances the safety of Large Language Models (LLMs) against adversarial attacks by explicitly increasing the distance between harmful and safe latent representations in the embedding space. This method, evaluated on open-source LLMs like Llama-2-7B-Chat-hf, Llama-3-8B-Instruct, Mistral-7B-Instruct, and Qwen-2.5-7B-Instruct, leverages the linear separability of harmful and harmless queries. ES2 introduces a Kullback-Leibler (KL) divergence regularization term to maintain the model's general capabilities on harmless inputs, preventing degradation. Experiments demonstrate that ES2 substantially improves Defense Success Rate (DSR) against embedding-level attacks (RepE, Soft Prompt, SCAV) and prompt-level attacks (AutoDAN, GCG), often by 15%-30% over baselines, while preserving comparable performance on benchmarks like MMLU-Pro and GPQA. The increased perturbation distance required for attacks leads to semantic collapse, resulting in incoherent or gibberish outputs.

Key takeaway

For research scientists developing or deploying LLMs, understanding and mitigating adversarial attacks is crucial. ES2 offers a robust defense by re-engineering the embedding space, making it significantly harder for both embedding-level and prompt-level attacks to succeed. You should consider implementing ES2's fine-tuning approach, particularly for open-source models, to establish a wider safety margin and ensure outputs remain coherent and harmless, even under aggressive adversarial pressure.

Key insights

Explicitly separating harmful and harmless embeddings in LLMs enhances safety without sacrificing general capabilities.

Principles

• Harmful and harmless embeddings are linearly separable.
• Large embedding perturbations destroy semantic integrity.

Method

ES2 fine-tunes LLMs by maximizing Euclidean distance between harmful and harmless embeddings at critical layers, regularized by KL divergence to preserve general capabilities. Training terminates if KL divergence exceeds a threshold.

In practice

• Apply ES2 to open-source LLMs for robust jailbreak defense.
• Target specific semantic emergence and terminal layers for fine-tuning.
• Use KL divergence to prevent general capability degradation.

Topics

• LLM Safety
• Embedding Space
• Adversarial Attacks
• Fine-tuning
• KL Regularization

Code references

• vinid/safety-tuned-llamas
• cassidylaidlaw/hidden-context
• andyzoujm/representation-engineering
• schwinnl/llm_embedding_attack
• SproutNan/AI-Safety_SCAV

Best for: Research Scientist, AI Researcher, AI Scientist, Machine Learning Engineer

Related on AIssential

• Adversarial Defence without Adversarial Defence: Enhancing Language Model Robustness via Instance-level Principal Component Removal — Transforming PLM embedding spaces to approximate Gaussian properties enhances adversarial robustness without costly adversarial training.
• F2LLM-v2: Inclusive, Performant, and Efficient Embeddings for a Multilingual World — F2LLM-v2 offers efficient, performant, and inclusive multilingual embeddings for over 200 languages.
• Learning Perturbations to Extrapolate Your LLM — Learnable continuous perturbations in embedding space significantly improve LLM out-of-domain extrapolation.
• Correcting Prompt Dependence in LLM Benchmarks: A Bayesian Hierarchical Model with Embedding-Space Clustering — Reliable LLM vulnerability evaluation requires a Bayesian hierarchical model with embedding-space clustering to quantify uncertainty and mitigate prompt bias.
• SafeReview: Defending LLM-based Review Systems Against Adversarial Hidden Prompts — SafeReview uses co-evolving Generator and Defender models to secure LLM-based peer review from adversarial hidden prompts.
• When Embedding-Based Defenses Fail: Rethinking Safety in LLM-Based Multi-Agent Systems — Embedding-based MAS defenses fail when attackers craft near-benign messages, necessitating token-level confidence signals for robustness.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CL updates on arXiv.org.