White Circle raises $11M to help companies secure and monitor AI model behavior
Summary
White Circle, operating as Pumpkin Intelligence Inc., has secured $11 million in seed funding to enhance AI model security and monitoring. The startup, founded by Denis Shilov, developed its technology after Shilov demonstrated in 2024 that many proprietary AI models could be jailbroken with a single prompt, bypassing safety measures to extract sensitive information or generate harmful content. The funding round included notable AI leaders from OpenAI, Anthropic, DeepMind, and DataDog. White Circle offers an API that uses specialized AI models to monitor both inputs and outputs in real time, detecting harmful content, hallucinations, prompt injection attacks, model drift, and malicious user activity based on custom policies. The company also published the "KillBench study," which involved over a million experiments across 15 AI models to identify hidden biases.
Key takeaway
For CTOs and VP of Engineering overseeing AI deployments, White Circle's $11 million funding highlights the critical need for advanced AI guardrails. You should evaluate your current AI security posture against sophisticated prompt injection and jailbreaking techniques. Consider integrating specialized AI monitoring solutions to protect sensitive data, prevent malicious use, and ensure model integrity, especially for user-facing agents handling critical information.
Key insights
White Circle raised $11M to secure AI models against jailbreaks and prompt injections using real-time input/output monitoring.
Principles
- AI models require robust guardrails.
- Real-time monitoring is crucial for AI security.
- AI model behavior can be unpredictable.
Method
White Circle employs specialized AI models via an API to track real-time inputs and outputs, detecting attacks, harmful content, and drift based on custom policies, improving accuracy through user feedback.
In practice
- Implement real-time input/output monitoring.
- Define custom policies for AI model behavior.
- Utilize user feedback to refine defensive models.
Topics
- AI Model Security
- AI Behavior Monitoring
- Prompt Injection Attacks
- AI Jailbreaking
- AI Guardrails
Best for: CTO, Investor, VP of Engineering/Data, AI Security Engineer, MLOps Engineer, Director of AI/ML
Related on AIssential
Editorial summary, takeaway, and curation by AIssential. Original article published by AI – SiliconANGLE.