Accelerating Diffusers and xDiT Image Generation with MXFP4 using AMD Quark on AMD Instinct™ MI350 GPUs

2026-07-06 · Source: AMD ROCm Blogs · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering · Depth: Advanced, long

Summary

AMD Quark, a high-performance quantization library, significantly accelerates Diffusers and xDiT FLUX.1-dev image generation on AMD Instinct™ MI350 GPUs using MXFP4 quantization. Benchmarks show MXFP4 ASM with "torch.compile" achieves a 1.92× speedup over BF16 eager and 1.41× speedup over BF16 "torch.compile" on a single MI350 GPU, reducing latency to 1.069 seconds per image while preserving CLIP quality (e.g., 31.84 CLIP score). Quark supports various numeric formats like MXFP4, FP8, and INT8, offers modular quantization flows, and integrates with inference pipelines via native AITER GEMM kernels. Furthermore, in a 2-GPU xFuser Ulysses setup, MXFP4 with "torch.compile" delivered a 1.23× throughput uplift over BF16, reaching 0.855 seconds per image at batch 16. This optimization enables efficient deployment of diffusion models with lower latency and memory footprint.

Key takeaway

For AI Engineers deploying diffusion models on AMD Instinct™ MI350 GPUs, you should integrate AMD Quark with MXFP4 quantization. This approach delivers up to 1.92× speedup over BF16 eager, reducing inference latency to 1.069 seconds per image while maintaining image quality. Consider using "torch.compile" alongside Quark for optimal performance, especially for multi-GPU setups where it offers a 1.23× throughput uplift.

Key insights

AMD Quark's MXFP4 quantization significantly boosts diffusion model inference speed on MI350 GPUs with minimal quality loss.

Principles

• Precision-aware quantization reduces inference cost.
• Block-level scaling in MXFP4 retains dynamic range.
• Native hardware support enhances low-precision GEMMs.

Method

Quark quantizes BF16 transformer linear layers to FP8/MXFP4 in-memory at load time, routing GEMMs to native AITER matrix-core kernels on MI300/MI350 GPUs.

In practice

• Configure per-layer quantization schemes with Quark.
• Use "RuntimeOptions(native_linear_mode=\"mxfp4\")" for ASM.
• Integrate Quark with Diffusers or xDiT pipelines.

Topics

• AMD Quark
• MXFP4 Quantization
• Diffusion Models
• AMD Instinct MI350
• xDiT
• Image Generation

Code references

• amd/quark
• amd/quark
• xdit-project/xDiT
• ROCm/aiter
• huggingface/diffusers

Best for: Machine Learning Engineer, AI Engineer, AI Hardware Engineer

Related on AIssential

• Advanced MXFP4 Quantization: Combining Fine-Tuned Rotations with SmoothQuant for Near-Lossless Compression — Combining fine-tuned online rotations and SmoothQuant scales significantly improves MXFP4 quantization accuracy for language models.
• Accelerating ComfyUI Workflows on AMD Instinct™ MI355X GPUs with ROCm — AMD Instinct MI355X GPUs significantly accelerate ComfyUI generative AI workflows over NVIDIA B200.
• ORBIS: Output-Guided Token Reduction with Distribution-Aware Matching for Video Diffusion Acceleration — SW-HW co-design with output-guided, distribution-aware token reduction significantly accelerates video Diffusion Transformers by exploiting temporal consistency.
• Lambda's MLPerf Inference v6.0: hardware leap, software maturity, research breakthrough — Hardware, software, and routing optimizations significantly boost AI inference performance and reduce latency.
• NVIDIA Ising Introduces AI-Powered Workflows to Build Fault-Tolerant Quantum Systems — NVIDIA Ising provides open AI models and tools to mitigate qubit noise and scale quantum computing.
• TileFuse: A Fused Mixed-Precision Kernel Library for Efficient Quantized LLM Inference on AMD NPUs — TileFuse enables efficient AWQ-style quantized LLM inference on AMD XDNA2 NPUs through fused mixed-precision kernels.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by AMD ROCm Blogs.