GenTI: Benchmarking LLMs for Autonomous IDPS Rule Generation for Unseen Attacks

2026-06-04 · Source: Artificial Intelligence · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Cybersecurity & Data Privacy · Depth: Expert, quick

Summary

GenTI (Generative Threat Intelligence) is an LLM-driven benchmark designed for the autonomous generation of Intrusion Detection and Prevention System (IDPS) rules, specifically targeting unseen attacks. It addresses the limitations of manually crafted, signature-driven rules and the lack of structured information in existing datasets for automatic rule synthesis. The accompanying GTI dataset aggregates over 150k detection and prevention rules from Snort, Suricata, and Emerging Threats, plus 50k YARA rules, all annotated with protocol behavior, payload signatures, CTI mappings, and actionable response types. GenTI's LLM-based pipeline transforms analyst prompts and payloads into deployable rules using structured prompt engineering, Chain-of-Thought (CoT) reasoning, and a Chain-of-Verification (CoVe) loop for validation. Evaluated on syntax accuracy, semantic similarity, CTI coverage, and security effectiveness, GenTI achieved a composite rule-quality score of 89.4%, 94.8% CTI coverage, improved unseen attack detection from 45% to 87.4%, and reduced false-positive rates from 8.5% to 2.3%. This establishes the first large-scale benchmark coupling rule-level CTI with LLM-based automation for adaptive IDPS.

Key takeaway

For security engineers developing or managing Intrusion Detection and Prevention Systems, GenTI demonstrates a path to significantly enhance adaptability against emerging threats. You should consider integrating LLM-driven automation with comprehensive Cyber Threat Intelligence to move beyond static, signature-based defenses. This approach can improve unseen attack detection from 45% to 87.4% and reduce false positives, making your IDPS more resilient and self-evolving.

Key insights

GenTI leverages LLMs and a comprehensive dataset to autonomously generate adaptive IDPS rules for unseen threats.

Principles

• Manual IDPS rules limit adaptability to zero-day threats.
• Structured CTI is crucial for automated rule synthesis.
• Verification loops enhance LLM-generated rule quality.

Method

An LLM-based pipeline transforms analyst prompts and payloads into deployable IDPS rules via structured prompt engineering, Chain-of-Thought reasoning, and a Chain-of-Verification loop.

In practice

• Integrate LLMs for dynamic IDPS rule updates.
• Utilize CTI-rich datasets for rule generation training.
• Implement CoVe for validating generated security rules.

Topics

• Intrusion Detection Systems
• Large Language Models
• Cyber Threat Intelligence
• Network Security Automation
• Zero-day Attacks
• Snort/Suricata Rules

Best for: Research Scientist, CTO, VP of Engineering/Data, AI Scientist, AI Security Engineer, Machine Learning Engineer

Related on AIssential

• GenTI: Benchmarking LLMs for Autonomous IDPS Rule Generation for Unseen Attacks — LLMs can autonomously generate CTI-aware IDPS rules for unseen attacks, significantly improving detection and reducing false positives.
• Spoon-feeding a Monster — Autonomous security testing requires separating LLM reasoning from deterministic execution to ensure ground truth and prevent hallucinations.
• SLMs, LLMs, and the Real Difference That Matters in DSPM — The real DSPM difference is not model size, but whether models predict fixed patterns or reason generatively.
• In-Context Autonomous Network Incident Response: An End-to-End Large Language Model Agent Approach — An LLM agent can autonomously handle network incident response by integrating perception, reasoning, planning, and action.
• Decoupled Smart Contract Audits: Lightweight LLM Framework via Distillation and Aggregation — Lightweight LLMs can achieve high accuracy in smart contract auditing by decoupling tasks and employing distillation and aggregation strategies.
• Continuous Offensive Security: The Line We've Been Walking — AI-driven offensive security is crucial for continuous defense against autonomous attackers and new AI-specific attack surfaces.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Artificial Intelligence.