Google Is Suing Chinese Scammers Who Are Using Gemini

2026-07-07 · Source: Schneier on Security · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Cybersecurity & Data Privacy · Depth: Expert, short

Summary

Google has filed a lawsuit against Outsider Enterprise, a Chinese cybercrime network operating a phishing-as-a-service scheme via Telegram. This group provided instructions and nearly 300 scam templates, leveraging Google's Gemini AI to generate fraudulent websites mimicking Google, YouTube, and government agencies like New York's E-ZPass. Google's legal action follows its efforts with AT&T, Verizon, and T-Mobile to block malicious text messages. Additionally, Google notes its on-device scam detection in Google Messages, an AI-powered feature, stops approximately 10 billion scam texts monthly, likely intercepting some of Outsider Enterprise's activity. The lawsuit highlights the increasing use of generative AI in sophisticated cybercrime operations.

Key takeaway

For AI Security Engineers evaluating generative AI for secure applications, you must recognize that current generative AI architecture is fundamentally incompatible with Zero-Trust principles. Do not rely on these models to self-regulate security boundaries, as their inherent "latent completion" and "Equal-Weight Failure" mechanisms mean they cannot recognize their own epistemic limits. Instead, implement external, continuous validation of intent and authorization at every layer.

Key insights

Generative AI's architectural design inherently conflicts with Zero-Trust security principles due to its inability to validate intent.

Principles

• LLMs cannot maintain epistemic caution, leading to Equal-Weight Failure.
• Generative models execute "latent completion," inventing default states to bridge information gaps.
• AI's architectural mandate to guess voids the Zero-Trust Evidence Contract.

Topics

• Gemini AI
• Phishing-as-a-Service
• Zero-Trust Architecture
• Generative AI Security
• Cybercrime
• Equal-Weight Failure

Best for: CTO, VP of Engineering/Data, Director of AI/ML, AI Security Engineer, Legal Professional, Research Scientist

Related on AIssential

• Data Privacy and Generative AI:The Truth About Common Security Promises — Cloud-based GenAI privacy promises often fall short due to retention, encryption limits, and legal overrides.
• Google sues Chinese cybercrime network that used Gemini to automate scams — Cybercrime networks are leveraging generative AI like Gemini to automate and scale sophisticated phishing-as-a-service operations.
• Securing AI Agents with Zero Trust — Zero Trust principles are essential for securing agentic AI systems against their expanded attack surfaces.
• 😺 Google sued the people spamming your phone — AI tools, including Google's Gemini, are being weaponized for sophisticated cybercrime, highlighting the urgent need for robust security measures and ethical AI deployment.
• Zero Trust for AI agents — Autonomous AI agents require a tailored Zero Trust framework to counter AI-accelerated threats and unique agentic security challenges.
• Perplexity Comet, agentic blabbering, and the shift-left failure — AI's internal reasoning and code analysis capabilities introduce new attack vectors and reveal hidden vulnerabilities in legacy systems.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Schneier on Security.