A Lifecycle and Application-Stack Survey of Large Language Model Vulnerabilities: Attacks, Risks, Defenses, and Open Problems

2026-06-30 · Source: Artificial Intelligence · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Cybersecurity & Data Privacy · Depth: Expert, quick

Summary

A comprehensive survey published on 2026-06-30 details vulnerabilities in large language model (LLM) systems, emphasizing that security risks extend beyond model weights to the entire application lifecycle and stack. The paper organizes attacks across eight stages: data collection, pretraining, post-training alignment, model packaging and supply chain, retrieval and memory, prompting and inference, tool/agent execution, and deployment/maintenance. For each stage, it analyzes attacker capabilities, security objectives, representative attacks, practical risks, evaluation practices, and defenses. The survey maps LLM-specific vulnerabilities to confidentiality, integrity, availability, safety, privacy, fairness, accountability, and agency-control objectives, highlighting how trust boundaries fail and untrusted data becomes executable instructions.

Key takeaway

For AI Security Engineers building or deploying LLM systems, you must shift your focus beyond model weights to the entire application lifecycle and stack. Evaluate security across all eight identified stages, from data collection to agent execution, to identify where trust boundaries fail. Prioritize defenses that address untrusted data becoming executable instructions and mitigate risks from delegated authority to prevent amplified model errors.

Key insights

LLM security must encompass the full application lifecycle and stack, not just model weights.

Principles

• Untrusted data can become executable instruction.
• Delegated authority amplifies model errors.
• Point defenses rarely compose effectively.

Method

The paper systematizes LLM vulnerabilities by organizing attacks across eight lifecycle and application-stack stages, analyzing risks and defenses for each.

In practice

• Focus on compositional security.
• Implement provenance-aware retrieval.
• Develop tool-call containment strategies.

Topics

• LLM Vulnerabilities
• Application Security
• AI Lifecycle Security
• Prompting Attacks
• Agent Security
• Supply Chain Security

Best for: CTO, AI Architect, Research Scientist, AI Security Engineer, AI Scientist, AI Engineer

Related on AIssential

• Tsinghua and Ant Group Researchers Unveil a Five-Layer Lifecycle-Oriented Security Framework to Mitigate Autonomous LLM Agent Vulnerabilities in OpenClaw — A five-layer security framework mitigates multi-stage vulnerabilities in autonomous LLM agents like OpenClaw.
• Data-Centric Benchmarking of Exploit Generation in LLMs: Understanding the Impact of Fine-Tuning — Data quality and structured evaluation are paramount for effective LLM-based exploit generation.
• CyberChainBench: Can AI Agents Secure Smart Contracts Against Real-World On-Chain Vulnerabilities? — LLM agents show promise but significant limitations in smart contract security tasks, especially patching.
• When Safe Skills Collide: Measuring Compositional Risk in Agent Skill Ecosystems — Individually safe agent skills can compose into unsafe installed skill sets, posing a significant compositional security risk.
• A Survey on Long-Term Memory Security in LLM Agents: Attacks, Defenses, and Governance Across the Memory Lifecycle — LLM agent memory's inherent malleability necessitates lifecycle-wide security governance for mnemonic sovereignty.
• The Emergence of Autonomous Penetration Capabilities in Large Language Model-Powered AI Systems — LLM-powered AI systems exhibit autonomous penetration capabilities, with success rates up to 69.3% in realistic evaluations, improving with model advancement.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Artificial Intelligence.