๐๏ธ This week on How I AI: How Intercom 2xโd their engineering velocity with Claude Code
Summary
Intercom doubled its engineering throughput in nine months by extensively integrating Claude Code, as detailed by Senior Principal Engineer Brian Scanlan. The company achieved this by treating its engineering organization like a product, instrumenting everything from skill invocations in Honeycomb to anonymized Claude Code sessions in S3, and building custom dashboards for performance insights. This velocity gain, measured by merged PRs per R&D employee, was built upon a mature foundation of CI/CD, comprehensive test coverage, and a high-trust culture. Intercom also implemented custom skills with hooks, such as a "Create PR" skill that enforces context-rich pull request descriptions, and developed self-improving AI agents to fix flaky tests. This approach has led to improved code quality and increased capacity for addressing technical debt, with the company now aiming for a 10x increase in throughput.
Key takeaway
For Directors of AI/ML and CTOs aiming to significantly boost engineering velocity, you should prioritize establishing a robust CI/CD pipeline and comprehensive test coverage before scaling AI tool adoption. Instrument your AI workflows with detailed telemetry to identify bottlenecks and opportunities for custom skill development. Empower your teams with explicit permission to experiment with AI, fostering a culture where accountability rolls up, enabling rapid iteration and a focus on agent-first work to tackle technical debt and accelerate product delivery.
Key insights
Intercom doubled engineering velocity in nine months by deeply integrating Claude Code, emphasizing telemetry and cultural permission.
Principles
- Treat your engineering organization like a product.
- AI magnifies existing strengths and weaknesses.
- Technical leadership must grant permission and take accountability.
Method
Instrument AI tool usage with telemetry (Honeycomb, S3) for visibility. Implement custom AI skills with hooks to enforce quality and automate workflows. Design agent-friendly product interfaces.
In practice
- Track skill invocations and session data for AI adoption insights.
- Build guardrails with custom skills for quality enforcement.
- Prioritize fixing foundational issues before scaling AI adoption.
Topics
- Claude Code
- Engineering Velocity
- AI Adoption Strategy
- Custom AI Skills
- Telemetry & Analytics
Best for: Software Engineer, Director of AI/ML, CTO
Related on AIssential
Editorial summary, takeaway, and curation by AIssential. Original article published by Lenny's Newsletter.