How AWS averted an AI supply chain disaster

2025-08-26 · Source: Welcome to the Artificial Intelligence Incident Database · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Cybersecurity & Data Privacy, Software Development & Engineering · Depth: Intermediate, medium

Summary

Amazon Web Services recently averted a significant software supply chain disaster involving its generative AI assistant, Amazon Q Developer for Visual Assistant Extension. The AWS security team discovered a GitHub token with excessive permissions within the CodeBuild configuration, allowing a threat actor to commit malicious code into the extension's open-source repository. This code was automatically included in a release. Fortunately, a syntax error prevented the malicious code from executing, thus averting potential data exfiltration, ransomware, or widespread disruption of IT and OT systems for AWS users. The incident, detailed in security bulletin AWS-2025-015, highlights the fragility of AI systems due to complex, interconnected infrastructure and the expanded attack surface presented by AI agents. AWS responded by revoking credentials, removing the code, and enhancing CodeBuild security.

Key takeaway

For MLOps Engineers or AI Security Engineers managing development environments, the Amazon Q incident underscores the critical need for proactive security. You must treat AI extensions and developer tools as privileged software, maintaining a complete inventory and actively monitoring for high-risk behaviors. Lock down your build pipelines with tightly scoped tokens and mandatory code reviews. Minimize runtime permissions through short-lived credentials and sandboxing to prevent destructive actions, ensuring luck isn't your security strategy.

Key insights

AI systems are only as secure as their weakest link, expanding the supply chain attack surface.

Principles

• AI fragility stems from system complexity.
• AI agents expand supply chain attack surface.
• Prompt injection is a real risk.

In practice

• Inventory all extensions with system access.
• Lock down build pipelines with scoped tokens.
• Minimize runtime permissions via sandboxing.

Topics

• AI Supply Chain Security
• Amazon Q Developer
• Software Supply Chain Attacks
• Prompt Injection
• GitHub Tokens
• CodeBuild Security

Best for: CTO, VP of Engineering/Data, Director of AI/ML, AI Security Engineer, MLOps Engineer, AI Engineer

Related on AIssential

• MCP Auto-Execution: From Git Clone to Cloud Compromise in Amazon Q VS Code Extension — AI coding tools' auto-execution of workspace configurations without consent poses a systemic risk for code execution and credential theft.
• Risk Ledger Secures $32M Series B - StartupHub.ai — Collective defense through a shared, continuously updated network assessment is crucial for modern supply chain security.
• The Week AI Agents Started Getting Weaponised — AI agents are rapidly automating high-skill cyberattack phases, significantly lowering the barrier for sophisticated threats.
• 13-hour AWS outage reportedly caused by Amazon's own AI tools — Agentic AI tools require robust access controls to prevent user-induced service disruptions.
• Your GitHub Token Is Now Worth More Than Your AWS Key — AI developers' credentials are now the primary target in supply chain attacks, shifting security focus from models to development environments.
• SE Radio 729: Garth Mollett on AI Supply Chain Security — AI supply chain security is complex due to merged control/data planes and probabilistic model behavior.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Welcome to the Artificial Intelligence Incident Database.