10 Open Projects for AI Security

2026-04-12 · Source: Turing Post · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Cybersecurity & Data Privacy, Robotics & Autonomous Systems · Depth: Intermediate, medium

Summary

AI security is no longer a "nice-to-have" and requires robust defensive measures, as modern AI models can exploit complex zero-days at scale. While Anthropic's Mythos model, a highly advanced defensive AI, remains closed to the public and is priced at $25 per million input tokens and $125 per million output tokens, several open-source tools and frameworks are available to enhance AI system safety. These include NVIDIA NeMo Guardrails for controlling LLM behavior, Promptfoo for automated vulnerability scanning and red teaming, LLM Guard for securing LLM interactions, NVIDIA garak for pre-deployment vulnerability scanning, DeepTeam for red teaming LLM systems, Meta's Llama Prompt Guard 2-86M for prompt injection detection, Google's ShieldGemma 2 for harmful image content detection, OpenGuardrails for agent security, Cupcake for policy-based agent action enforcement, and Meta's CyberSecEval 3 for visual prompt injection evaluation.

Key takeaway

For CTOs and VPs of Engineering responsible for AI deployments, prioritize integrating comprehensive AI security from the outset. Your teams should adopt open-source tools like NeMo Guardrails, Promptfoo, and LLM Guard to establish automated guardrails, continuous red teaming, and robust vulnerability scanning, mitigating risks from advanced AI-driven attacks and ensuring system integrity.

Key insights

Robust AI security is critical, necessitating open-source tools for defense against advanced model-driven exploits.

Principles

• AI security must be integrated early, not as a late-stage add-on.
• Automated defenses are essential against scalable AI exploits.

Method

Implement a multi-layered defense strategy using programmable guardrails, automated red teaming, vulnerability scanning, and real-time monitoring for LLM and agentic systems.

In practice

• Use NeMo Guardrails for LLM behavior control.
• Integrate Promptfoo for CI/CD-based security testing.
• Deploy Llama Prompt Guard 2-86M for prompt injection detection.

Topics

• AI Security
• LLM Guardrails
• Prompt Injection Defense
• LLM Red Teaming
• Vulnerability Scanning

Code references

• NVIDIA-NeMo/Guardrails
• promptfoo/promptfoo
• protectai/llm-guard
• NVIDIA/garak
• confident-ai/deepteam

Best for: CTO, VP of Engineering/Data, Director of AI/ML, AI Security Engineer, MLOps Engineer, AI Engineer

Related on AIssential

• OpenAI to acquire Promptfoo — OpenAI's acquisition of Promptfoo integrates AI security testing and evaluation into its enterprise AI platform, Frontier.
• The Emergence of Autonomous Penetration Capabilities in Large Language Model-Powered AI Systems — LLM-powered AI systems exhibit autonomous penetration capabilities, with success rates up to 69.3% in realistic evaluations, improving with model advancement.
• Project Lightwell brings open source security into the AI era — AI both exacerbates open source security risks through advanced attack chaining and offers solutions through augmented security efforts.
• Should you let OpenClaw pen test your system? Plus: Cybersecurity for ephemeral software — AI agents offer significant potential for cybersecurity, but require careful management and integration to be effective and safe.
• OpenSSF CTO on Building Trust in Open Source with AI — OpenSSF aims to secure the open source ecosystem through proactive measures, AI tools, and global collaboration.
• 😺 OpenAI built a crypto thief — AI agents, particularly OpenAI's GPT-5.3-Codex, are highly effective at exploiting smart contract vulnerabilities.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Turing Post.