The future of AI is community driven and open

2026-07-23 · Source: Cloud Native Computing Foundation · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Cloud Computing & IT Infrastructure · Depth: Advanced, short

Summary

NVIDIA, through its Senior Director Erin A. Boyd, asserts that Kubernetes is the de facto operating system for AI, with 66% of organizations using it for generative AI inference workloads. Despite this, only 7% of organizations deploy AI models daily, indicating a significant operational gap in continuous production, particularly for inference and agentic workloads. NVIDIA is deepening its commitment to the cloud native community and CNCF, investing \$4 million over three years and joining the CNCF Governing Board, to address the lack of first-class GPU support within Kubernetes. Key contributions include the NVIDIA GPU Dynamic Resource Allocation (DRA) Driver, now upstream in Kubernetes SIG-Node, which enables real-time, on-demand GPU allocation and features like MIG device sharing. Additionally, the KAI Scheduler, accepted as a CNCF Sandbox project, provides advanced gang scheduling for large AI clusters. The Kubernetes AI Conformance Program has also expanded to 31 certified platforms, ensuring consistent AI infrastructure. NVIDIA champions an open, community-driven approach, believing it fosters greater innovation and broader adoption than proprietary platforms.

Key takeaway

For MLOps Engineers and AI Architects deploying large-scale AI on Kubernetes, the current static GPU resource management leads to significant inefficiencies. You should evaluate and integrate NVIDIA's open-source contributions like the GPU Dynamic Resource Allocation (DRA) Driver and the KAI Scheduler. Adopting these community-governed tools will enable dynamic, on-demand GPU allocation, efficient gang scheduling for multi-tenant clusters, and verifiable AI infrastructure, directly improving your operational efficiency and scalability for continuous AI workloads.

Key insights

Scalable, continuous AI on Kubernetes demands open, community-driven GPU orchestration and standardized infrastructure.

Principles

• Foundational AI infrastructure should be a community asset.
• Open, interoperable platforms build larger ecosystems.
• GPU management belongs in Kubernetes standards.

In practice

• Implement NVIDIA GPU DRA Driver for dynamic GPU allocation.
• Adopt KAI Scheduler for large-scale AI cluster scheduling.
• Use Kubernetes AI Conformance Program for infrastructure verification.

Topics

• Kubernetes
• GPU Orchestration
• Cloud Native
• NVIDIA
• AI Infrastructure
• Dynamic Resource Allocation
• KAI Scheduler

Code references

• NVIDIA/nvidia-container-toolkit
• NVIDIA/gpu-operator
• kubernetes-sigs/nvidia-dra-driver-gpu
• NVIDIA/KAI-Scheduler
• cncf/k8s-ai-conformance

Best for: CTO, VP of Engineering/Data, Director of AI/ML, MLOps Engineer, AI Engineer, AI Architect

Related on AIssential

• Is Kubernetes the DeFacto OS for Enterprise AI infrastructure? — Kubernetes is essential for enterprise AI, unifying GPU orchestration, resource efficiency, and scalable inference across diverse workloads.
• Understanding dynamic resource allocation in Kubernetes — Kubernetes DRA provides granular, declarative GPU allocation, surpassing older Device Plugin limitations.
• Breaking free of a single datacenter: Practical geo-distributed AI operations with the k0smos platforms — Geo-distributed AI operations are viable using Kubernetes-native multi-cluster orchestration for heterogeneous hardware and dynamic resource management.
• Validate Kubernetes for GPU Infrastructure with Layered, Reproducible Recipes — AI Cluster Runtime simplifies Kubernetes configuration for AI workloads via validated, version-locked, and reproducible recipes.
• Day 15 of 100 Days of GenAI for DevOps: GenAI on Kubernetes — Kubernetes requires specific components like Device Plugins to enable GPU support for AI workloads.
• KubeCon + CloudNativeCon, OpenInfra Summit and PyTorch Conference Unite in China to Scale AI — Converging cloud native, open infrastructure, and PyTorch communities standardizes scalable, production-grade AI platforms.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Cloud Native Computing Foundation.