Hardware-Rooted AI Security That Won’t Slow You Down

2026-07-02 · Source: NVIDIA Technical Blog · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Cybersecurity & Data Privacy · Depth: Advanced, short

Summary

NVIDIA Confidential Computing (CC) offers a secure and performant solution for agentic AI, protecting enterprise data and proprietary model weights during active inference. Engineered into Blackwell GPUs, including the NVIDIA RTX PRO 6000, HGX B200, and HGX B300, CC provides a hardware root of trust with a private signing key fused at manufacturing. Before execution, the NVIDIA Remote Attestation Service (NRAS) verifies the GPU's hardware report and CPU TEE measurements (AMD SEV-SNP or Intel TDX) against a reference integrity manifest. Benchmarks using the Qwen 3.5 397B-A17B model at FP8 precision on HGX B300 demonstrated inference performance nearly identical, up to 98%, to non-CC solutions, with minimal throughput and time per output token overhead. Innovations like CC-safe autotuner timing in FlashInfer and Async D2H copy worker in SGLang optimize performance.

Key takeaway

For MLOps Engineers or AI Architects concerned with securing proprietary AI models and sensitive data during inference, NVIDIA Confidential Computing on Blackwell GPUs provides a robust, hardware-rooted solution. You can achieve strong data privacy and regulatory compliance, such as HIPAA or GDPR, without compromising performance, as benchmarks show up to 98% throughput compared to non-CC setups. Evaluate integrating CC into your production AI workloads to leverage its attestation and encryption capabilities.

Key insights

NVIDIA Confidential Computing secures AI inference with hardware-rooted trust and minimal performance overhead.

Principles

• Hardware root of trust is foundational.
• Attestation validates TEE integrity pre-execution.
• Optimize inference frameworks for CC.

Method

Remote attestation verifies a GPU hardware report and CPU TEE measurements against a reference integrity manifest via NRAS, enabling secure deployment of model decryption keys into a Confidential VM.

In practice

• Deploy AI workloads on Blackwell GPUs.
• Integrate NRAS for remote attestation.
• Configure SGLang with CUDA graph support.

Topics

• NVIDIA Confidential Computing
• AI Security
• Hardware Root of Trust
• Remote Attestation
• Blackwell GPUs
• AI Inference Performance

Best for: CTO, VP of Engineering/Data, Director of AI/ML, AI Security Engineer, MLOps Engineer, AI Architect

Related on AIssential

• Building a Zero-Trust Architecture for Confidential AI Factories — Confidential computing with TEEs and CoCo enables secure, zero-trust AI factories by encrypting data and models during execution.
• Intel vPro Security Drives New AI PC Innovations with the Security Ecosystem — Intel's vPro Security solutions integrate silicon-level AI and hardware encryption for advanced endpoint protection.
• Confidential AI with GPU Acceleration: Bounce Buffers Offer a Solution Today — Bounce buffers enable secure data transfer between CPU and GPU TEEs for Confidential AI workloads.
• Nvidia launches Dynamo 1.0 AI inference operating system — Nvidia has launched Dynamo 1.0, an open-source "operating system" for large-scale AI inference, now in production and available to developers globally.
• Introducing NVIDIA Fleet Intelligence for Real-Time GPU Fleet Visibility and Optimization — NVIDIA Fleet Intelligence provides comprehensive, real-time monitoring and integrity verification for large-scale GPU and CPU fleets.
• Apple teams up with Google, Nvidia to expand private cloud capabilities — Apple's expanded Private Cloud Compute leverages partnerships to enhance AI capabilities while maintaining privacy.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by NVIDIA Technical Blog.