Engineering Insights: How Internal Optimizations Led to Comet Cost Intelligence

2026-07-08 · Source: Comet · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering · Depth: Intermediate, medium

Summary

Comet developed its Cost Intelligence product by addressing internal AI spending challenges, which now helps customers reduce AI costs by 10-40% without sacrificing performance. The company first optimized its Opik MCP server, discovering that many shipped tools were unused, increasing prompt sizes and costs. By rebuilding the MCP with a smaller set of composable CRUD-style operations and self-discovery tools, prompt overhead dropped by roughly 100x, making the interface more capable. Second, Comet optimized its skill library, increasing skill selection rates from 2% to 35% by improving discoverability and reducing monthly AI spend by 5% by merging redundant skills. These efforts underscore the importance of continuous measurement and optimization of AI usage, as the ecosystem constantly evolves with models like Claude Code and Codex.

Key takeaway

For MLOps Engineers tasked with optimizing AI spend, continuously monitoring your model's actual usage of tools and skills is crucial. You should instrument your AI systems to identify underutilized components, like unused MCP tools or redundant skill library content, and refactor them. This approach, exemplified by Comet's 10-40% cost reductions, allows you to significantly cut expenses and improve model performance by reducing unnecessary context, rather than imposing blunt token limits.

Key insights

Measuring AI usage and costs reveals inefficiencies, enabling significant optimizations in model performance and spend without sacrificing capability.

Principles

• Unused tools inflate prompt context and cost.
• Measure actual usage of AI components.
• Simplify interfaces for model reasoning.

Method

Instrument AI usage to collect cost data, analyze for underutilized or redundant components like MCP tools or skills, then refactor based on usage to reduce context and improve efficiency. Continuously monitor for new inefficiencies.

In practice

• Rebuild MCPs with composable CRUD operations.
• Update skill trigger conditions for discoverability.
• Merge or eliminate redundant skills.

Topics

• AI Cost Optimization
• MLOps
• Prompt Engineering
• LLM Tool Use
• Skill Libraries
• Observability

Best for: AI Engineer, MLOps Engineer, Director of AI/ML

Related on AIssential

• Six Choices Every AI Engineer Has to Make (and Nobody Teaches) — Production AI involves critical trade-offs where decision costs often manifest far from the initial choice.
• Measuring the Hidden Costs of AI-Generated Insights: A Data Analyst’s Guide to Autonomous Pipeline… — Autonomous AI pipelines accrue hidden, unbudgeted costs over time, impacting long-term ROI.
• 5 Practical Ways to Reduce AI Costs (TCO) in the Enterprise — Optimizing enterprise AI costs requires architectural strategies beyond just technical solutions.
• AI Is Fast. AI Projects Are Slow. Let's Fix That. — Standardized AI pipeline frameworks are crucial for reliable, scalable, and cost-efficient production deployments.
• How to Navigate the Shift from Prompt-Based Tools to Workflow-Driven AI — The proliferation of specialized AI tools creates workflow friction; unified platforms offer a solution by integrating models and context.
• 8 Ways I Reduced My AI API Bill by 60 Percent Without Any User Noticing Any Difference — Significant AI API cost reductions are achievable by eliminating non-value-adding expenditures without quality degradation.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Comet.