RDC and RocProfiler Compared to DCGM for Commonly Used Metrics

2026-07-07 · Source: AMD ROCm Blogs · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Cloud Computing & IT Infrastructure, Software Development & Engineering · Depth: Intermediate, long

Summary

The article introduces AMD's ROCm Data Center tool (RDC) and RocProfiler for GPU performance monitoring, comparing its metrics to NVIDIA's DCGM. RDC, released July 07, 2026, integrates ROCm-SMI for hardware status, RAS for reliability data (e.g., ECC errors), and RocProfiler for runtime metrics like active cycles and FLOPs. It uses a daemon ("rdcd") and a command-line tool ("rdci") for interactive or automated monitoring. The post categorizes common metrics into hardware health (throttling, XGMI traffic), error monitoring (ECC), runtime utilization (VALUBusy, GPU_UTIL_PERCENT, TENSOR_PERCENT), and data type activity (FP16, FP32, FP64 FLOPs). It demonstrates how to use "rdci dmon" with field IDs like "100" (GPU_CLOCK) and "600" (ECC_CORRECT) and explains the MI308 GPU hierarchy (4 XCDs, 320 SIMD units) for metric interpretation.

Key takeaway

For MLOps Engineers validating AMD GPU clusters or tuning applications, RDC offers a critical command-line tool for quick performance triage. You can efficiently monitor hardware health, inter-GPU traffic, and ECC errors without full profiling. Use RDC to identify throttling, low compute utilization, or incorrect data type usage early. This helps you pinpoint whether performance issues stem from system limits or kernel inefficiencies, guiding your optimization efforts effectively.

Key insights

AMD's RDC provides a unified command-line interface for GPU monitoring, consolidating hardware, reliability, and runtime performance metrics.

Principles

• GPU performance triage starts with hardware health.
• Occupancy and VALUBusy measure different aspects.
• Error counters distinguish hardware from software issues.

Method

RDC uses "rdcd" daemon for telemetry collection and "rdci" command-line tool to query specific metrics via field IDs (e.g., "rdci dmon -u -e 100,600 -i 0").

In practice

• Check "RDC_HEALTH_POWER_THROTTLE_TIME" for performance drops.
• Monitor XGMI traffic for multi-GPU workload validation.
• Verify data type activity (FP16, FP32) for precision modes.

Topics

• ROCm Data Center Tool
• GPU Performance Monitoring
• RocProfiler
• AMD GPUs
• Hardware Telemetry
• ECC Error Detection

Best for: Machine Learning Engineer, AI Engineer, MLOps Engineer

Related on AIssential

• Introduction to profiling tools for AMD hardware — AMD offers diverse profiling tools to optimize application performance across its CPU and GPU architectures.
• AMD Device Metrics Exporter v1.4.2: Enhanced Observability, Deeper RAS Insights, and Smarter GPU Telemetry for Modern HPC & AI Clusters — DME v1.4.2 enhances GPU observability with process-level context, structured error reporting, and detailed performance constraint metrics.
• Taking on CUDA With ROCm: ‘One Step After Another’ — AMD's ROCm software stack is rapidly maturing, leveraging open-source tools and community engagement to challenge Nvidia's CUDA dominance.
• Fixing GPU Starvation in Large-Scale Distributed Training — GPU underutilization often stems from data I/O and format transformation bottlenecks, not model complexity.
• NVIDIA NVbandwidth: Your Essential Tool for Measuring GPU Interconnect and Memory Performance — NVbandwidth measures GPU memory transfer performance to optimize CUDA applications and validate system configurations.
• torch-nvenc-compress: GPU NVENC silicon as a PCIe bandwidth multiplier — PCA + pure-ctypes Video Codec SDK wrapper. Parallel-path overlap measured at 67% of theoretical max on a real GEMM + encode workload. [P] — GPU NVENC/NVDEC silicon can act as a PCIe bandwidth multiplier by compressing tensor data in parallel with compute.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by AMD ROCm Blogs.