Meta Contractors Posed as Teens to Prompt Rival Chatbots About Suicide, Sex, and Drugs

2026-06-29 · Source: WIRED - Ai · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Fundamental Awareness, medium

Summary

Meta contractors, working on a project internally known as Cannes and managed by Covalen, posed as minors to test rival chatbots like OpenAI's ChatGPT, Google's Gemini, and Character.AI. Active as recently as April 21, the project involved creating dummy under-18 accounts and submitting over 45,000 prompts and images, including pills, knives, and nooses, designed to push safety systems on high-risk topics such as suicide, self-harm, eating disorders, sex, and drugs. The companies being tested were unaware of this activity. While Meta defended it as "industry-standard practice" for AI safety benchmarking, experts and former contractors expressed concerns about the project's scale, opacity, and potential for anticompetitive behavior. Attorneys found the prompts did not solicit child sexual abuse material, but the testing likely violated the competitors' terms of service, which prohibit unauthorized safety testing and bypassing safeguards.

Key takeaway

For AI ethicists evaluating industry safety practices, this incident highlights the critical need for transparency in competitor benchmarking. You should scrutinize claims of "industry-standard" testing, especially when covert methods and fake identities are involved. Ensure your organization's safety evaluations adhere to explicit ethical guidelines and respect competitor terms of service to avoid legal and reputational risks. This case underscores the governance gray zones where safety claims can mask anticompetitive behavior.

Key insights

Meta contractors secretly tested rival chatbots as minors with high-risk prompts, raising ethical and legal questions about "industry-standard" AI safety.

Principles

• Covert testing with fake minor accounts exceeds "industry standard" evaluation.
• Blending safety and competitor benchmarking creates governance gray zones.
• Unauthorized testing often violates competitor terms of service.

Method

Contractors created dummy under-18 accounts, sent high-risk written prompts and images to rival chatbots (ChatGPT, Gemini, Character.AI), and recorded responses in spreadsheets, often pushing safety system boundaries.

Topics

• AI Safety Benchmarking
• Chatbot Ethics
• Terms of Service Violations
• Competitive Intelligence
• Contractor Labor
• Meta Platforms

Best for: CTO, Executive, VP of Engineering/Data, AI Ethicist, Legal Professional, Tech Journalist

Related on AIssential

• Meta reportedly used contractors to test rival AI chatbots — Meta's use of contractors to test rival AI safety exposes industry-wide content moderation challenges and a shift to AI-driven review.
• Meta secretly tested ChatGPT, Gemini, and Character.AI with thousands of minor-perspective crisis prompts — Covertly testing AI chatbots with minor-perspective crisis prompts raises significant ethical, legal, and terms of service concerns.
• Meta’s Worker Surveillance Tests EU Rules on AI and Labor — Meta's employee surveillance for AI training tests regulatory limits and highlights the value of human behavioral data for agentic models.
• Position: Behavioural Assurance Cannot Verify the Safety Claims Governance Now Demands — Behavioral AI assurance cannot verify the deep safety claims demanded by current governance frameworks.
• Sam Altman is “the face of evil” for not reporting school shooter, says lawyer — OpenAI is accused of negligence for failing to report a violent user, prioritizing privacy and IPO valuation over public safety.
• # Australia’s AI Ethics Principles Are a Start. Evidence Is the Real Test. — Practical AI ethics are challenged by dominant norm bias, which benefits powerful actors and hinders equitable outcomes.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by WIRED - Ai.