Occupancy Math on the AMD MI355X GPU (CDNA4): A From-First-Principles Guide

2026-07-07 · Source: AMD ROCm Blogs · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering · Depth: Expert, extended

Summary

A guide to occupancy math on the AMD Instinct™ MI355X GPU (CDNA4, gfx950) details how kernel performance is determined by resource limits rather than simply maximizing occupancy. The MI355X GPU features 256 Compute Units, 512 VGPRs per lane shared by regular and accumulator registers, and 160 KB of Local Data Share (LDS) per CU. Occupancy is capped by VGPRs, SGPRs, LDS, or workgroup/barrier slots. The article demonstrates that while CDNA3 kernels were often LDS-bound at 25% occupancy, CDNA4's larger 160 KB LDS shifts the bottleneck to VGPRs, achieving 50%. Crucially, a microbenchmark shows that high Instruction-Level Parallelism (ILP) can sustain ~97% of the ~5 PFLOP/s MXFP8 matrix peak even at 12% occupancy, outperforming higher occupancy with lower ILP.

Key takeaway

For GPU kernel engineers optimizing for the AMD MI355X GPU, prioritize feeding the Matrix Core over maximizing occupancy. Focus on increasing Instruction-Level Parallelism (ILP) by packing more independent accumulator tiles per wave, and use the largest possible register-resident tiles without spilling. Leverage the 160 KB LDS for deeper software-pipelined prefetch to keep the Matrix Core saturated, even if it means lower occupancy. Measure matrix-engine and VALU utilization, not just occupancy percentage, to guide your tuning decisions.

Key insights

Maximizing GPU occupancy is often counterproductive; prioritize Instruction-Level Parallelism (ILP) and larger register-resident tiles for peak performance.

Principles

• Occupancy is set by four resource limiters.
• VGPRs and AccVGPRs share a 512-entry file.
• ILP often outperforms Thread-Level Parallelism (TLP).

Method

Calculate occupancy ceiling by dividing hardware budgets (VGPRs, SGPRs, LDS, workgroup slots) by kernel resource usage, converting units, and taking the minimum.

In practice

• Use "llvm-objdump" for exact resource usage.
• Account for register allocation granularity (e.g., 8 VGPRs).
• Spend LDS on pipeline depth, not accumulators.

Topics

• AMD MI355X GPU
• CDNA4 Architecture
• GPU Occupancy
• Instruction-Level Parallelism
• Matrix Core
• Kernel Optimization
• MXFP8 GEMM

Code references

• triton-lang/triton

Best for: Machine Learning Engineer, AI Engineer, AI Hardware Engineer

Related on AIssential

• GPU Scaling from 1 to 261120 threads Part 2 — GPU performance scaling plateaus due to L2 bandwidth saturation and memory-bound operations, despite latency hiding.
• AMD Instinct™ GPUs MLPerf Inference v6.0 Submission — AMD's MI355X GPU and ROCm stack deliver competitive MLPerf Inference v6.0 results, especially for LLMs and multi-node setups.
• GlobalFoundries: First Chipmaker to Support Open Standard for AI Scale Up — GlobalFoundries' early OCI MSA support and advanced silicon photonics enable open, multi-vendor AI data center interconnects, mitigating GPU supply risks.
• Introducing hipThreads: A C++ - Style Concurrency Library for AMD GPUs — hipThreads enables incremental GPU acceleration of C++ `std::thread` code using familiar concurrency patterns and persistent multi-fiber execution.
• Maximize AI Infrastructure Throughput by Consolidating Underutilized GPU Workloads — GPU partitioning via MIG or time-slicing significantly improves resource utilization and throughput in Kubernetes AI deployments.
• AI in Multiple GPUs: Understanding the Host and Device Paradigm — Understanding Host-Device interaction, asynchronous execution, and CUDA Streams is fundamental for optimizing GPU performance.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by AMD ROCm Blogs.