The Product Engineer Company: How Portkey Works and Builds Its Product
Summary
Portkey, an AI infrastructure company soon to be acquired by Palo Alto Networks, operates with 32 employees, including 24 engineers, primarily from India. They focus on "plumbing infrastructure for AI," providing an AI gateway, observability, guardrails, governance, and prompt management. Their Node.js and TypeScript-heavy stack supports services like their Gateway and Control Panel. Portkey empowers engineers with end-to-end ownership, expecting them to manage features from development to customer support. They utilize custom AI agents, including a task-tracking agent for customer interactions and a "Dagger" pricing agent that monitors 1600+ LLMs across 80+ providers for pricing and API changes, automatically generating pull requests. The company prioritizes scalable solutions for production traffic, processing trillions of tokens daily for over 100 enterprise customers, and avoids features like AI evals that divert from their core mission. Their engineering culture balances deep reliability with rapid prototyping, and approximately 40% of their code is AI-generated.
Key takeaway
For Directors of AI/ML building infrastructure-heavy products, Portkey's model demonstrates the value of empowering engineers with end-to-end product ownership. You should foster a culture where engineers manage features from conception to support, ensuring deep technical understanding drives product decisions. Consider adopting a "barbell" organizational structure to balance reliability with rapid innovation, and strategically deploy custom AI agents to automate critical, repetitive tasks like API monitoring and updates, freeing your team for complex problem-solving.
Key insights
Portkey's success stems from deep product ownership by engineers and a disciplined focus on scalable AI infrastructure.
Principles
- Engineers should have end-to-end product ownership.
- Prioritize production reliability and scalability over feature breadth.
- Adapt roadmaps to strategic themes, not fixed long-term plans.
Method
Portkey uses custom AI agents, like "Dagger" for pricing, to automate monitoring and updates across 1600+ LLMs and 80+ providers, generating PRs for changes.
In practice
- Integrate AI agents for automated monitoring and PR generation.
- Store context in markdown files within repos for AI agent interpretation.
- Empower engineers with full feature lifecycle ownership.
Topics
- AI Infrastructure
- AI Gateway
- Engineering Culture
- End-to-End Ownership
- Custom AI Agents
- LLM Monitoring
Best for: Investor, CTO, VP of Engineering/Data, AI Engineer, MLOps Engineer, Director of AI/ML
Related on AIssential
Editorial summary, takeaway, and curation by AIssential. Original article published by Engineering Leadership.