Hackers can use 9 of the most popular AI tools to assemble massive botnets

2026-07-08 · Source: AI - Ars Technica · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Cybersecurity & Data Privacy, Robotics & Autonomous Systems · Depth: Expert, medium

Summary

Researchers have unveiled HalluSquatting, a novel pull-based prompt injection attack capable of assembling massive botnets and executing large-scale distributed denial-of-service (DDoS) attacks. This attack exploits the inherent tendency of large language models (LLMs) to hallucinate resource identifiers, such as repository or skill names, when prompted by users. It targets nine popular AI coding assistants and agents, including Cursor, Gemini CLI, and GitHub Copilot, which routinely access high-privilege command lines. The attack works by predicting identifiers LLMs are most likely to hallucinate (e.g., up to 85% for repositories, 100% for trending skills), registering these "squatted" names, and embedding malicious instructions like reverse shells. This allows attackers to indiscriminately infect numerous devices at scale, a capability previously unachieved by prompt injection methods. The flaw is foundational across six major LLMs, including Gemini-2.5-flash and GPT-5.2, with hallucination rates reaching 92.4% for newer resources.

Key takeaway

For AI Security Engineers deploying or managing LLM-powered coding assistants, you must recognize that current guardrails are insufficient against scalable prompt injection. Your teams should implement robust validation for all external resource resolution, as LLMs predictably hallucinate identifiers for new or trending packages. Assume agents will be fooled; prioritize resilience and manual verification of resource locations to prevent large-scale botnet formation or data exfiltration.

Key insights

LLMs' predictable hallucination of resource identifiers enables scalable, pull-based prompt injection attacks like HalluSquatting.

Principles

• LLMs inherently struggle to distinguish trusted from untrusted instructions.
• Hallucination patterns for resource identifiers are predictable across major LLMs.
• Newer, trending resources are significantly more prone to LLM hallucination.

Method

HalluSquatting involves predicting LLM-hallucinated resource identifiers, registering them, and seeding them with malicious instructions (e.g., reverse shells) in repositories or skill files. Coding agents then execute these commands.

In practice

• Identify common LLM hallucination patterns for resource resolution.
• Register "squatted" resource names that LLMs are likely to fabricate.
• Embed malicious code or instructions within registered resources.

Topics

• HalluSquatting
• Prompt Injection
• LLM Security
• AI Coding Assistants
• Botnets
• Typosquatting

Best for: CTO, VP of Engineering/Data, Director of AI/ML, AI Security Engineer, AI Engineer, AI Scientist

Related on AIssential

• Spoon-feeding a Monster — Autonomous security testing requires separating LLM reasoning from deterministic execution to ensure ground truth and prevent hallucinations.
• How indirect prompt injection attacks on AI work - and 6 ways to shut them down — Indirect prompt injection is a top LLM security risk, weaponizing AI without direct user input.
• The Range Shrinks, the Threat Remains: Re-evaluating LLM Package Hallucinations on the 2026 Frontier-Model Cohort — LLM package hallucination rates have converged but still pose a significant, model-agnostic supply-chain security risk.
• Hacking Meta’s AI Chatbot — LLM chatbots are inherently untrustworthy for security-critical applications, as attack vectors cannot be blocked comprehensively.
• Evaluating Prompting-Based Defenses Against Domain-Camouflaged Injection Attacks — Paraphrasing retrieved content is the most effective prompting-based defense against domain-camouflaged injection attacks.
• LLM APIs are now the #1 attack surface in AI systems. — LLM APIs are the top AI attack surface due to executable intent inputs, requiring specialized security beyond traditional methods.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by AI - Ars Technica.