The Eticas AI Risk Taxonomy: Open Infrastructure for Operationalizing AI Audits
Summary
The Eticas AI Risk Taxonomy v2.0.0 provides an open infrastructure designed to operationalize AI audits, addressing the fragmentation in risk evaluation where many existing taxonomies catalog risks without offering execution methods. This framework demonstrates a bridge from risk concept to graded finding, exemplified by measuring PII leakage on GPT-4-0314. This specific test showed disclosure rates of 0%, 51%, and 84% as adversarial conditioning increased, leading to a subcategory grade of E with a SYSTEMIC pattern. The taxonomy organizes 76 active subcategories across 10 categories and 20 sub-groups, mapping to 18 external frameworks. Its conceptual layer is open under CC BY 4.0 with stable URIs and SKOS/JSON-LD distributions, while the methodology calibration forms the practitioner layer.
Key takeaway
For AI Security Engineers tasked with auditing high-stakes AI systems, you should adopt operationalized risk taxonomies like Eticas AI Risk Taxonomy v2.0.0. This framework moves beyond mere risk cataloging by providing a concrete methodology to measure, calibrate, and grade risks such as PII leakage, as demonstrated on GPT-4-0314. Integrate its open conceptual scaffold and mapping to 18 external frameworks to standardize your audit processes and produce defensible findings.
Key insights
The Eticas AI Risk Taxonomy operationalizes AI audits by bridging risk cataloging with concrete, measurable evaluation methods.
Principles
- AI auditing requires operationalization, not just risk cataloging.
- Separate risks from their surfacing mechanisms.
- An open-core model supports shared conceptual scaffolds.
Method
The Eticas method involves turning a risk into a test run against a real system, measuring values, calibrating severity, and assigning a defensible grade.
In practice
- Measure PII leakage on LLMs like GPT-4-0314.
- Use the Eticas taxonomy's 76 subcategories.
- Map findings to 18 external compliance frameworks.
Topics
- AI Risk Taxonomy
- AI Auditing
- Operationalization
- PII Leakage
- GPT-4
- Compliance Frameworks
- Open Infrastructure
Best for: CTO, VP of Engineering/Data, Director of AI/ML, AI Security Engineer, AI Ethicist, Research Scientist
Related on AIssential
Editorial summary, takeaway, and curation by AIssential. Original article published by Artificial Intelligence.