Baseten: "We've Never Lost Our Top Customers”

2026-06-01 · Source: No Priors: AI, Machine Learning, Tech, & Startups · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Cloud Computing & IT Infrastructure, Corporate Strategy & Leadership · Depth: Intermediate, quick

Summary

Baseten highlights a critical distinction in the cloud computing market: "GPUs as a Service" is perceived as a commodity with low customer stickiness, whereas "Inference with Software Layer" demonstrates exceptional customer retention. The company reports zero churn among its top 30 customers and a 400% annual Net Dollar Retention (NDR) for its software-enhanced inference offerings. This strong performance underscores the strategic importance of the software layer in providing value beyond raw compute. Baseten asserts that access to inference compute, particularly when bundled with a robust software layer, is evolving into a significant strategic advantage for businesses, including major research labs.

Key takeaway

For AI Product Managers evaluating inference infrastructure, recognize that raw GPU compute is a commodity. Your focus should shift towards solutions that integrate a robust software layer, as this drives significant customer stickiness and higher Net Dollar Retention. Prioritize vendors offering comprehensive inference platforms over basic GPU-as-a-service to secure a strategic advantage and ensure long-term value for your AI applications.

Key insights

Software layers transform commodity GPU compute into sticky, strategic inference solutions.

Principles

• GPU-as-a-service is a commodity.
• Software layers drive customer stickiness.
• Inference compute is a strategic advantage.

In practice

• Integrate software with inference compute.
• Focus on NDR for sticky services.

Topics

• GPU as a Service
• AI Inference
• Software Layer
• Customer Retention
• Strategic Advantage
• Cloud Computing

Best for: CTO, VP of Engineering/Data, AI Architect, Director of AI/ML, AI Product Manager, Entrepreneur

Related on AIssential

• The AI Industry Spent Billions Chasing Faster Chips. Inference Startup Kog Says They Were Solving the Wrong Problem. — Software optimization can enable standard GPUs to match specialized AI hardware performance for real-time inference.
• Most AI teams treat compute as a commodity. It's not. — AI compute quality, not just quantity, dictates project success, efficiency, and unit economics for frontier models.
• [AINews] The Inference Inflection — The AI industry faces an "inference inflection" driving massive compute demand and platform evolution for coding agents.
• GPU Clouds, Aggregators, and the New Economics of AI Compute — GPU cloud providers offer diverse services and pricing, necessitating careful evaluation based on security, managed services, and workload portability.
• Perplexity, CoreWeave Deal Boosts Inferencing — The AI market is shifting focus from training to inference, driving specialized cloud partnerships for real-time AI applications.
• Lambda at NVIDIA GTC 2026: our thoughts — The AI infrastructure market prioritizes execution and scalable, production-ready systems over theoretical possibilities.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by No Priors: AI, Machine Learning, Tech, & Startups.