NVIDIA NVbandwidth: Your Essential Tool for Measuring GPU Interconnect and Memory Performance

2026-04-14 · Source: NVIDIA Technical Blog · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering · Depth: Intermediate, medium

Summary

NVIDIA NVbandwidth is a CUDA-based tool designed to measure memory bandwidth and latency across various memory copy patterns in single-GPU, multi-GPU, and multi-node NVIDIA systems. It utilizes either copy engine (CE) or kernel copy methods to report current measured bandwidth, offering insights into data transfer performance between CPU memory and GPU memory, and GPU memory to GPU memory. The tool supports comprehensive unidirectional, bidirectional, multi-GPU, and multi-node tests, along with latency testing. NVbandwidth is topology-agnostic, working across NVLINK, NVLink C2C, or PCIe interconnects, and provides flexible output options including plain text and JSON. It requires a CUDA-enabled NVIDIA GPU, CUDA toolkit (version 11.X+ for single-node, 12.3+ for multi-node), compatible NVIDIA display driver, C++17 compiler, CMake 3.20+, and Boost program options library.

Key takeaway

For ML infrastructure engineers and system architects evaluating or optimizing GPU deployments, NVbandwidth provides critical metrics for data transfer performance. You should integrate NVbandwidth into your validation workflows to benchmark new hardware, identify bottlenecks in existing systems, and perform regression testing after software or driver updates. This ensures your CUDA applications achieve optimal data movement and overall system efficiency.

Key insights

NVbandwidth measures GPU memory transfer performance to optimize CUDA applications and validate system configurations.

Principles

• Memory bandwidth is critical for GPU application performance.
• Data transfer patterns impact overall system efficiency.

Method

NVbandwidth measures performance by enqueuing a spin kernel, then a start event, multiple memcpy iterations, and a stop event, ensuring overhead exclusion.

In practice

• Use NVbandwidth to diagnose CUDA application bandwidth bottlenecks.
• Compare bandwidth across GPUs to evaluate system upgrades.
• Run regression tests to detect performance changes after updates.

Topics

• NVIDIA NVbandwidth
• GPU Performance Optimization
• Memory Bandwidth Measurement
• Multi-GPU Systems
• Multi-Node GPU Deployments

Code references

• NVIDIA/nvbandwidth

Best for: Machine Learning Engineer, AI Architect, MLOps Engineer

Related on AIssential

• Accelerating Data Processing with NVIDIA Multi-Instance GPU and NUMA Node Localization — Data localization via NVIDIA MIG can significantly boost GPU performance under power constraints.
• AI in Multiple GPUs: Understanding the Host and Device Paradigm — Understanding Host-Device interaction, asynchronous execution, and CUDA Streams is fundamental for optimizing GPU performance.
• Quantum Computing Reaches an Inflection Point With NVIDIA NVQLink | GTC 2026 — NVQLink bridges quantum and classical computing, essential for hybrid quantum applications and practical quantum computer development.
• Achieving Peak System and Workload Efficiency on NVIDIA GB200 NVL72 with Slurm Block Scheduling — NVIDIA GB200 NVL72's rack-scale NVLink coherence requires Slurm's topology/block plugin for optimal scheduling.
• Performance Profiling on AMD GPUs - Part 4: Fortran OpenMP Offload Edition — Profiling Fortran OpenMP offload on AMD GPUs requires a systematic workflow to identify memory-bound kernels and optimize HSA_XNACK and workgroup sizes.
• AMD Device Metrics Exporter v1.4.2: Enhanced Observability, Deeper RAS Insights, and Smarter GPU Telemetry for Modern HPC & AI Clusters — DME v1.4.2 enhances GPU observability with process-level context, structured error reporting, and detailed performance constraint metrics.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by NVIDIA Technical Blog.