Multi-turn RL with Structural and Performance Aware Rewards for CUDA Kernel Generation

2026-07-23 · Source: Takara TLDR - Daily AI Papers · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering · Depth: Expert, quick

Summary

CudaPerf introduces a reflective Reinforcement Learning (RL) framework designed for optimized CUDA kernel generation, addressing limitations in existing RLVR approaches that primarily focus on outcome-based signals. This framework integrates both verifiable execution rewards and structural code-aware rewards, considering parallelization features like memory coalescing, occupancy, Arithmetic Intensity, and synchronization patterns. CudaPerf operates in two stages: an offline pairwise ranking module and an online RL training phase that jointly optimizes for correctness, performance, and structural efficiency. It also employs iterative refinement with execution feedback. Evaluated on a new dataset of 2.9k C to CUDA and 1k PyTorch to CUDA programs, CudaPerf significantly outperforms baselines like Qwen-3-32B and CUDA Agent, achieving up to 5X and 3.32X speedup, and 17% and 7% correctness improvements, respectively.

Key takeaway

For Machine Learning Engineers optimizing CUDA kernels, CudaPerf offers a robust approach to improve both performance and correctness. You should consider integrating structural code properties, such as memory coalescing and Arithmetic Intensity, into your reward functions for RL-based code generation. This method can yield substantial speedups and higher correctness rates compared to relying solely on execution outcomes, potentially streamlining your GPU programming efforts.

Key insights

CudaPerf enhances LLM code generation by integrating structural code properties with execution rewards in a multi-turn RL framework.

Principles

• Combine outcome-based and structural rewards for code optimization.
• Iterative refinement improves generated code candidates.

Method

CudaPerf uses an offline pairwise ranking module for candidate distinction, followed by online RL training to jointly optimize correctness, performance, and structural efficiency, enhanced by iterative execution feedback.

In practice

• Generate optimized CUDA kernels from C or PyTorch.
• Utilize structural metrics like memory coalescing for code quality.

Topics

• CUDA Kernel Generation
• Reinforcement Learning
• Code Optimization
• LLM Code Generation
• Memory Coalescing
• CudaPerf

Best for: Research Scientist, AI Engineer, AI Scientist, Machine Learning Engineer

Related on AIssential

• Multi-turn RL with Structural and Performance Aware Rewards for CUDA Kernel Generation — CudaPerf enhances LLM code generation by integrating structural and performance-aware rewards in a multi-turn RL framework.
• OptiML: An End-to-End Framework for Program Synthesis and CUDA Kernel Optimization — OptiML unifies LLM-driven code generation with hardware-aware search to optimize CUDA kernels for performance.
• Kernel Fusion in NVIDIA CUDA: Optimizing Memory Traffic and Launch Overhead — Kernel fusion combines GPU operations to reduce memory traffic and launch overhead for faster execution.
• Optimizing CUDA like a Human: Micro-Profiling Tools as Expert Surrogates for LLM-Based GPU Kernel Optimization — LLMs can optimize GPU kernels by integrating with micro-profiling tools and iterative search, achieving significant speedups and energy efficiency.
• Topping the GPU MODE Kernel Leaderboard with NVIDIA cuda.compute — The `cuda.compute` library enables Python developers to achieve near "speed-of-light" GPU performance with CUB primitives.
• AscendKernelGen: A Systematic Study of LLM-Based Kernel Generation for Neural Processing Units — Domain-specific reasoning and rigorous evaluation are crucial for LLM-based NPU kernel generation.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Takara TLDR - Daily AI Papers.