Extract More Kernel Performance with NVIDIA CompileIQ Auto-Tuning

2026-05-26 · Source: NVIDIA Technical Blog · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering · Depth: Advanced, long

Summary

NVIDIA CompileIQ, released with CUDA 13.3, is an AI-powered compiler auto-tuning framework designed to extract maximum performance from specific GPU workloads. It utilizes evolutionary and genetic algorithms to optimize NVIDIA general purpose GPU compilers by exploring internal parameters like register allocation and instruction scheduling, which are not publicly exposed. This addresses the "90% problem" where a small fraction of code, such as GEMMs and attention kernels in LLM inference, dominates compute time. CompileIQ generates an advanced controls file (ACF) that tailors compiler configurations for critical kernels, offering performance gains on top of existing optimizations. The framework supports multi-objective optimization, allowing teams to balance runtime, compile time, and power consumption, and has shown up to 15% performance improvements in production workloads at Meta. It is available as a Python package and ensures IP protection by processing workloads locally.

Key takeaway

For AI Engineers and MLOps teams optimizing GPU-accelerated workloads, if you have exhausted traditional performance tuning methods, NVIDIA CompileIQ provides a novel approach. You should integrate this AI-powered auto-tuning framework to discover workload-specific compiler configurations, potentially achieving up to 15% additional performance on critical kernels like GEMMs and attention. Define your objective function to measure what matters most—runtime, power, or compile time—and leverage CompileIQ to generate reproducible advanced controls files for deployment.

Key insights

NVIDIA CompileIQ leverages AI-driven evolutionary algorithms to automatically fine-tune GPU compiler configurations for workload-specific peak performance.

Principles

• Compiler default heuristics are "good across the board" but not "optimal for your workload".
• Small performance gains in critical code sections yield outsized overall application performance.
• Compiler internals can be a new optimization lever after exhausting other tuning methods.

Method

Install CompileIQ, define a Python objective function to benchmark kernel performance, configure evolutionary search parameters (e.g., generations, pool size), and run the search to generate an optimized advanced controls file (ACF).

In practice

• Apply CompileIQ to high-impact kernels like GEMM and attention in LLM inference.
• Use multi-objective optimization to balance runtime, compile time, and power consumption.
• Version control generated ACFs for reproducible and portable compiler optimizations.

Topics

• NVIDIA CompileIQ
• GPU Performance Optimization
• Compiler Auto-tuning
• Evolutionary Algorithms
• LLM Inference
• Multi-objective Optimization

Code references

• triton-lang/triton
• pytorch/helion
• nvidia/compileiq
• NVIDIA/CompileIQ

Best for: Machine Learning Engineer, AI Engineer, MLOps Engineer

Related on AIssential

• Speeding up GPU kernels by 38% with a multi-agent system — Multi-agent systems can autonomously optimize complex software, achieving significant performance gains in weeks.
• AMD GPU Programming From Beginner to Expert (Part 1) - TensorDescriptor in Composable Kernel (CK) — Composable Kernel's TensorDescriptor uses hierarchical transforms to efficiently manage complex multi-dimensional data layouts on AMD GPUs.
• Develop High-Performance GPU Kernels in C++ with NVIDIA CUDA Tile — CUDA Tile C++ simplifies high-performance GPU kernel development by abstracting low-level details through a portable tile-based programming model.
• Accelerating AI-Powered Chemistry and Materials Science Simulations with NVIDIA ALCHEMI Toolkit-Ops — NVIDIA ALCHEMI Toolkit-Ops accelerates atomistic simulations by providing GPU-optimized, batched operations for chemistry and materials science.
• Nautilus: An Auto-Scheduling Tensor Compiler for Efficient Tiled GPU Kernels — Nautilus automates tensor kernel optimization from math specifications, outperforming existing compilers.
• From Naive to Near-Peak: Building High-Performance GEMM Kernels with Gluon — Achieving near-peak GPU kernel performance requires explicit hardware control and a profiling-driven, iterative optimization methodology.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by NVIDIA Technical Blog.