The new Wild West of AI kids’ toys

2026-05-09 · Source: AI - Ars Technica · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Regulatory Affairs & Government Relations, Mental Health & Psychological Support · Depth: Fundamental Awareness, long

Summary

The market for AI-powered children's toys is rapidly expanding, with over 1,500 companies registered in China by October 2025 and products like Huawei's Smart HanHan selling 10,000 units in its first week. Despite this growth, the category remains largely unregulated, raising significant concerns among consumer groups and researchers. Tests by organizations like PIRG found toys powered by models like OpenAI's GPT-4o providing age-inappropriate content, including instructions on lighting matches or discussing sex and drugs. A University of Cambridge study involving the Curio Gabbo AI toy with children aged 3-5 identified developmental issues related to non-intuitive conversational turn-taking, hindered social play, and the toy's inability to convey it is not a living social partner. Additionally, issues like "dark patterns" encouraging isolation and addiction, along with data privacy concerns, have been reported, prompting calls for stricter regulations and legislative action in the US and EU.

Key takeaway

For product managers and engineers developing children's AI products, you must prioritize robust, age-appropriate guardrails and independent safety assessments from the outset. Your designs should explicitly communicate the toy's non-human nature and avoid features that mimic social partnership or encourage addictive behaviors, ensuring compliance with emerging regulations like California's proposed moratorium and the EU's AI Act.

Key insights

Unregulated AI toys pose developmental risks and safety concerns for children due to age-inappropriate content and flawed interaction models.

Principles

• AI models for adults are unsuitable for children's toys.
• Independent, multidisciplinary testing is crucial before market release.
• Toys must convey they are not living social partners.

Method

The University of Cambridge study monitored 14 children aged 3-5 interacting with a commercially available AI toy, the Curio Gabbo, to assess developmental impacts on language, social play, and relational integrity.

In practice

• Build open-source, local voice AI toys for controlled inputs.
• Prioritize "dumb" toys for young children.
• Disable conversational AI features via parental controls.

Topics

• AI-Powered Toys
• Child Safety Regulations
• Developmental Psychology
• Data Privacy
• AI Model Vetting

Code references

• akdeb/Elato-Local

Best for: CTO, VP of Engineering/Data, Director of AI/ML, AI Ethicist, Policy Maker, AI Product Manager

Related on AIssential

• AI Toys Reach Children Before Privacy and Safety Rules Catch Up — AI toys are deployed faster than privacy, safety, and child development research can establish adequate safeguards.
• Politicians couldn't keep kids off social media. They’ll try again with AI. — Political and industry leaders are converging on age-based AI regulation, learning from social media's impact on children.
• We Make Sure Kids’ Pajamas Are Safe. Why Not Their AI? — Independent testing and clear safety standards are crucial for protecting children from harmful AI products.
• Report calls for AI toy safety standards to protect young children — GenAI toys pose developmental and safety risks for young children, necessitating urgent regulation and transparent standards.
• States Step in Where Congress Stalls on AI Safeguards for Kids — States are proactively regulating AI chatbots for minors due to federal legislative delays and increasing harm incidents.
• Grok is undressing children — can the law stop it? — Grok's AI-generated deepfakes of minors highlight critical failures in content moderation and platform accountability.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by AI - Ars Technica.