Beyond rate limits: scaling access to Codex and Sora

2026-02-11 · Source: OpenAI News · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering · Depth: Advanced, medium

Summary

OpenAI has developed a real-time access engine to scale usage for its Codex and Sora products, addressing limitations of traditional rate limits and usage-based billing. This new system, detailed in a February 13, 2026 engineering post by Jonah Cohen, integrates rate limits, real-time usage tracking, and credit balances into a single "decision waterfall" model. It allows users to seamlessly transition from rate-limited access to credit-based usage within the same request, preventing frustrating hard stops. The in-house built system prioritizes real-time correctness, reconcilability, and user trust, ensuring transparent auditing of usage events, monetization events, and balance updates. This architecture aims to protect user momentum by providing continuous access and provably correct billing.

Key takeaway

For AI Product Managers overseeing high-demand services, you should consider adopting a hybrid access model that blends real-time rate limits with credit-based fallback. This approach, exemplified by OpenAI's system for Codex and Sora, maintains user engagement by preventing abrupt service interruptions while ensuring fair resource allocation and transparent billing. Prioritize building an auditable system that guarantees correctness to foster user trust and support continuous product usage.

Key insights

A hybrid access model combining real-time rate limits and credit-based usage enhances user experience and system scalability.

Principles

• Prioritize user momentum over strict limits.
• Provable correctness builds user trust.
• Integrate access logic into a unified system.

Method

Implement a "decision waterfall" for access, evaluating rate limits then credits within a single request path, with asynchronous, idempotent balance updates.

In practice

• Audit usage, monetization, and balance events separately.
• Use idempotency keys to prevent double-debiting.
• Tolerate minor balance update delays for correctness.

Topics

• Access Control Systems
• Real-time Billing
• Rate Limiting
• Codex and Sora
• Distributed Systems Architecture

Best for: AI Product Manager, Product Manager, CTO, Software Engineer, MLOps Engineer, AI Architect

Related on AIssential

• OpenAI Codex referral program rewards users with extra rate resets — OpenAI's new flexible rate limits for Codex empower users and open doors for future monetization through banked resets.
• AI Agents and the Fight for Customer Data — Centralized, open data infrastructure is crucial for AI agents, despite vendor data access restrictions.
• Rate Limiter System Design: Token Bucket, Leaky Bucket, Scaling — Token Bucket algorithm is the industry standard for robust API rate limiting, balancing simplicity and effectiveness.
• Is OpenAI the New GitHub? — AI infrastructure and business models are rapidly evolving across major tech companies.
• A practical guide for platform teams managing shared AI deployments — Rate limiting protects model health; quota reservations ensure critical workload priority in shared AI deployments.
• Meta is adding ridiculous ‘rate limits’ and a soft paywall to its smart glasses — Meta is imposing server-independent rate limits on smart glasses features, raising questions about fair use and subscription models for owned hardware.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by OpenAI News.