Detecting Vulnerabilities in Agent Skills with SkillSpector: From Green Checkmark to Real Security Judgment

2026-07-22 · Source: Towards Data Science · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Cybersecurity & Data Privacy · Depth: Advanced, extended

Summary

NVIDIA's SkillSpector, an open-source scanner for AI agent skills, employs a two-layer approach to detect vulnerabilities, combining deterministic static analysis with LLM-based semantic checks. While its static layer is fast and effective at catching obvious threats, including plain-English prompt injections, it exhibits an approximately 80 percent false-positive rate on legitimate skills, flagging routine API calls as risks. The tool's numeric score, ranging from 0 to 100, can be misleading; for instance, a malicious honeypot scored 100, while a complex automation skill's score shifted from 31 to 27 after semantic analysis, clarifying actual risks. This highlights that agent skills, which treat all content as instructions, are a new software supply chain vulnerability, with studies showing 26.1 percent of 42,447 skills containing prompt injection and Snyk reporting 76 percent of 3,984 skills as malicious.

Key takeaway

For AI Security Engineers approving agent skills, do not rely solely on a scanner's single numeric score. You must review the full scan output as a list of distinct behaviors, understand the skill's stated purpose to differentiate features from threats, and consider the author's reputation and trigger specificity. This approach, leveraging semantic analysis beyond static checks, is critical to avoid approving credential stealers or desensitizing your team to scanner warnings.

Key insights

Agent skills' instruction-centric nature makes prompt injection trivial, necessitating semantic analysis beyond static code scans.

Principles

• Static skill scanning prioritizes recall, generating many false positives.
• Skill risk scores are lossy; context and stage-specific findings are key.
• Semantic analysis is crucial for detecting reworded or intent-based threats.

Method

SkillSpector combines deterministic checks (patterns, AST, taint tracking, YARA, CVE) with three LLM-based discovery passes (semantic injection, developer intent, quality/policy), followed by a meta-analyzer to filter findings and calculate a contextualized score.

In practice

• Use static scans for initial skill ingestion and queue ordering.
• Apply LLM-based passes at the approval gate for precision.
• Baseline known findings to focus on new, critical risks.

Topics

• Agent Skills
• AI Security
• Prompt Injection
• SkillSpector
• LLM Security
• Vulnerability Detection

Code references

• nvidia/skillspector

Best for: CTO, VP of Engineering/Data, Director of AI/ML, AI Security Engineer, MLOps Engineer, AI Engineer

Related on AIssential

• Seeing Is Not Screening: Multimodal Hidden Instruction Attacks on Agent Skill Scanners — Multimodal hidden instructions exploit text-only agent skill scanners, necessitating execution-grounded multimodal defense.
• Agent Skills Work but the Research Shows Most Teams Are Building Them Wrong — Curated, focused agent skills improve AI performance, but require structured organization and security diligence to scale effectively.
• Agent Skills for Large Language Models: Architecture, Acquisition, Security, and the Path Forward — Agent skills modularize LLM capabilities, extending functionality dynamically while introducing significant security challenges.
• tech-leads-club / agent-skills — Agent Skills provides a secure, validated, and extensible library of capabilities for AI coding agents.
• NVIDIA / SkillSpector — AI agent skills pose significant security risks, necessitating pre-installation vulnerability scanning.
• ClawHub Security Signals: When VirusTotal, Static Analysis, and SkillSpector Disagree — AI agent skill security requires layered governance due to significant scanner disagreement across attack surfaces.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Towards Data Science.