Topping the GPU MODE Kernel Leaderboard with NVIDIA cuda.compute

2026-02-18 · Source: NVIDIA Technical Blog · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering · Depth: Advanced, quick

Summary

The NVIDIA `cuda.compute` library provides a high-level, Pythonic API for device-wide CUB primitives, enabling Python developers to write high-performance GPU code without resorting to C++ kernels. This library allows for custom data types and operators defined directly in Python, utilizing just-in-time (JIT) compilation and link-time optimization to achieve near "speed-of-light" (SOL) performance comparable to CUDA C++. An NVIDIA CCCL team demonstrated the library's effectiveness by achieving the most first-place finishes across NVIDIA B200, H100, A100, and L4 GPU architectures in the GPU MODE kernel competition, outperforming many custom C++ implementations for standard parallel operations like Sort, PrefixSum, VectorAdd, Histogram, and Grayscale.

Key takeaway

For AI Engineers building GPU-accelerated Python libraries or custom pipelines, `cuda.compute` offers a critical advantage. You can now leverage NVIDIA's highly optimized CUB primitives directly within Python, eliminating the need for complex C++ bindings and achieving "speed-of-light" performance. This accelerates your development cycle through rapid iteration and ensures production-grade performance for common parallel operations, allowing you to focus on novel algorithms rather than re-implementing standard kernels.

Key insights

The `cuda.compute` library enables Python developers to achieve near "speed-of-light" GPU performance with CUB primitives.

Principles

• High-level abstractions can deliver SOL performance.
• JIT compilation enables rapid iteration with C++ performance.

Method

`cuda.compute` JIT compiles specialized kernels and applies link-time optimization to expose architecturally tuned CUB primitives directly in Python.

In practice

• Use `cuda.compute` for standard parallel primitives.
• Define custom data types and operators in Python.
• Install via `pip install cuda-cccl[cu13]` or `conda install -c conda-forge cccl-python`.

Topics

• NVIDIA cuda.compute
• GPU Programming
• CUDA C++
• Python for ML
• Performance Optimization

Code references

• NVIDIA/cccl

Best for: AI Engineer, AI Scientist, Software Engineer, Machine Learning Engineer, Research Scientist

Related on AIssential

• NVIDIA CUDA 13.3 Enhances GPU Development with Tile Programming in C++, Compiler Autotuning, and Python Updates — CUDA 13.3 streamlines GPU development with high-level abstractions and performance optimizations across C++ and Python ecosystems.
• Using Accelerated Computing to Live-Steer Scientific Experiments at Massive Research Facilities — GPU-accelerated computing enables real-time analysis and steering of exascale scientific experiments, drastically reducing data processing times.
• AI in Multiple GPUs: Understanding the Host and Device Paradigm — Understanding Host-Device interaction, asynchronous execution, and CUDA Streams is fundamental for optimizing GPU performance.
• How NVIDIA Turned GPUs into Supercomputers — NVIDIA GPUs, initially for gaming, democratized scientific computing by offering supercomputer-level performance.
• Evaluating CUDA Tile for AI Workloads on Hopper and Blackwell GPUs — CuTile offers high productivity and peak performance on datacenter Blackwell, but lacks portability and consistent performance across other architectures.
• A Guide to Understanding GPUs and Maximizing GPU Utilization — GPU bottlenecks often stem from inefficient data pipelines, not compute, requiring optimized data transfer and processing.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by NVIDIA Technical Blog.