Amazon kills internal AI leaderboard after employees gamed it with pointless tasks

2026-05-29 · Source: The Decoder · Field: Business & Management — Corporate Strategy & Leadership, Operations & Process Management · Depth: Fundamental Awareness, quick

Summary

Amazon discontinued its internal AI ranking system, "Kirorank," on May 29, 2026, after employees exploited it by directing AI agents to perform "pointless tasks" on the Kiro developer platform. This behavior inflated their scores and significantly increased the company's cloud costs. Senior Vice President Dave Treadwell acknowledged the dashboard's "good intentions" but cited the unintended cost escalation. This incident occurred as Amazon aims for over 80 percent weekly AI developer usage and plans to invest approximately \$200 billion in 2026, primarily in AI infrastructure. A similar issue with employees chasing AI usage scores was reported at Meta. Amazon is now shifting its focus to tracking "normalized deployments" of AI-generated code, emphasizing actual utility over raw token consumption.

Key takeaway

For Directors of AI/ML implementing AI adoption strategies, recognize that gamified usage metrics like raw token consumption can lead to significant cost overruns and unproductive activity. You should prioritize tracking "normalized deployments" or other metrics that directly measure the utility and integration of AI-generated code into production. This shift ensures your investments drive actual business value, not just inflated internal scores.

Key insights

Gamified AI usage metrics can inadvertently drive up costs and encourage unproductive activity, necessitating a focus on useful deployments.

Principles

• Metrics must align with value, not just activity.
• Unintended consequences arise from gamification.
• Focus on useful output, not raw consumption.

In practice

• Avoid raw token consumption as a metric.
• Track AI-generated code deployments.
• Prioritize useful AI outputs.

Topics

• AI Adoption Metrics
• Gamification Risks
• Cloud Cost Optimization
• Kiro Platform
• Normalized Deployments
• AI Infrastructure Spending

Best for: CTO, AI Product Manager, Product Manager, Director of AI/ML, VP of Engineering/Data, Executive

Related on AIssential

• How AI is changing Software Engineering: A Conversation with Gergely Orosz, @pragmaticengineer — Measuring AI token output can lead to counterproductive "token maxing" behavior among engineers.
• Quests, token leaderboards, and a skills marketplace: The elite AI adoption playbook | John Kim (Sendbird) — Sendbird's internal AI platform and gamified adoption strategy empower non-technical teams to build and deploy AI-powered solutions.
• Google Cracked AI Monetization. Meta, Microsoft, and Amazon Haven't. — Clear AI monetization strategies that demonstrate value creation beyond efficiency are crucial for investor confidence.
• How to get your whole team using AI — Tracking internal AI token usage reveals adoption patterns and identifies top AI consumers.
• We are in the gaslighting phase of AI adoption — Companies are offloading AI adoption risks onto employees, creating a "gaslighting" environment where AI's immaturity is downplayed.
• The Pulse: ‘Tokenmaxxing’ as a weird new trend — Gamified AI token usage metrics can lead to massive waste and counterproductive behaviors within organizations.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by The Decoder.