How Intercom 2X'd engineering velocity with Claude Code | Brian Scanlan
Summary
Intercom's R&D department has doubled its throughput, measured by pull requests (PRs) per R&D head, in nine months by fully embracing AI-assisted development, primarily using Claude Code. This initiative, driven by a company-wide "AI-first" strategy and a CTO-set goal, has not only increased velocity but also improved code quality, as indicated by a Stanford research group's analysis. The company manages this transformation by treating its internal AI adoption like a product, implementing extensive telemetry with Honeycomb and Snowflake for skill usage and session data analysis. They've also developed an internal skills repository, distributed via IT systems, to standardize and enhance developer workflows, exemplified by a "flaky specs" skill that autonomously fixes recurring test issues.
Key takeaway
For Directors of AI/ML and VPs of Engineering aiming to significantly boost R&D velocity, you should commit to an "agent-first" development paradigm. Prioritize investing in AI tools and internal skill development, even if initial costs are high, and establish robust telemetry to monitor usage and quality. This approach can lead to substantial throughput gains and improved code quality, transforming your team's capacity to tackle technical debt and deliver features faster.
Key insights
Intercom doubled R&D throughput and improved code quality by fully integrating AI-assisted development and treating internal AI adoption as a product.
Principles
- Treat internal AI adoption as a product.
- Prioritize speed and investment over immediate cost optimization.
- Foster a high-trust culture for AI integration.
Method
Implement comprehensive telemetry for AI skill usage and session data. Develop an internal, IT-distributed skills repository. Enforce quality standards for AI-generated outputs (e.g., PR descriptions) via automated hooks and LLM judges.
In practice
- Instrument internal AI skills with telemetry (e.g., Honeycomb).
- Analyze raw AI session data for organizational insights.
- Invest in fixing tech debt using AI-driven automation.
Topics
- Engineering Velocity
- Claude Code
- AI Adoption Strategy
- Developer Experience
- Code Quality Improvement
Best for: AI Engineer, Director of AI/ML, Software Engineer
Related on AIssential
Editorial summary, takeaway, and curation by AIssential. Original article published by How I AI.