How AI is changing Software Engineering: A Conversation with Gergely Orosz, @pragmaticengineer

· Source: AI Engineer · Field: Technology & Digital — Software Development & Engineering, Artificial Intelligence & Machine Learning · Depth: Intermediate, extended

Summary

The concept of "token maxing" has emerged in large tech companies like Meta, Microsoft, and Salesforce, where engineers are incentivized or pressured to increase their AI token output. This practice, initially a joke, has evolved into a serious concern as token counts are sometimes used in performance evaluations and promotion decisions, leading engineers to generate tokens even through inefficient means, such as asking AI to summarize documentation instead of reading it. Some companies, like Salesforce, even set minimum monthly AI spend targets. This trend is driven by leadership's desire for AI adoption, stemming from a belief that increased AI usage correlates with innovation and productivity, despite evidence suggesting individual productivity gains are not always realized. The internal push for AI has also led to significant investment in custom internal AI infrastructure, with companies like Uber and Shopify building bespoke solutions to integrate AI into their existing monorepos and tooling, often justified by the need to handle vast codebases and attract top talent.

Key takeaway

For engineering leaders evaluating AI integration strategies, be wary of productivity metrics like AI token count, as they can incentivize superficial usage rather than genuine innovation. Your teams might engage in "token maxing" to meet targets, creating junk output and fostering a culture of fear. Instead, focus on tangible impact and provide robust internal AI infrastructure that genuinely enhances workflows, allowing engineers to leverage AI effectively without resorting to performative metrics.

Key insights

Measuring AI token output can lead to counterproductive "token maxing" behavior among engineers.

Principles

Method

Companies are building custom internal AI infrastructure, including custom coding agents and MCP gateways, to integrate AI into existing systems and handle large codebases that off-the-shelf solutions cannot.

In practice

Topics

Best for: Software Engineer, AI Engineer, Director of AI/ML

Related on AIssential

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by AI Engineer.