KroQuant: Kronecker-Structured Block Transforms for Efficient Post-Training Quantization of Diffusion Transformers

2026-07-23 · Source: Machine Learning · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Computer Vision · Depth: Expert, quick

Summary

KroQuant is a post-training quantization (PTQ) method designed to address the severe output quality degradation observed when quantizing diffusion transformers (DiTs) to W4A4, a problem caused by activation outliers. Traditional solutions involve invertible linear transforms, but normalization layers necessitate online execution, making inference computation cost a critical design constraint. Existing methods like SmoothQuant offer low cost but compromise accuracy, while Hadamard transforms improve accuracy at higher online costs, and learned full-$d$ transforms are computationally prohibitive. KroQuant introduces a learned Kronecker-structured invertible transform applied to each 32-element activation block. This approach stores less than half the parameters of per-channel scaling, executes as small tensor-core GEMMs, and achieves up to 14% faster quantizer kernel performance than SmoothQuant on an MI350 GPU. It also incorporates offline LoRaQ weight calibration to manage residual quantization error. Benchmarked on PixArt-$Σ$, SANA, and FLUX.1-schnell at W4A4 (MXFP4e2), KroQuant yields image quality superior to SVDQuant and LoRaQ on MJHQ-30K and SDCI.

Key takeaway

For Machine Learning Engineers deploying diffusion transformers, if you are struggling with W4A4 post-training quantization degrading model quality or incurring high inference costs, KroQuant offers a compelling solution. This method enables efficient 4-bit quantization by addressing activation outliers with Kronecker-structured transforms, achieving faster kernel performance and superior image quality compared to existing techniques. You should evaluate KroQuant to significantly reduce the memory footprint and accelerate inference of your DiT models without compromising output fidelity.

Key insights

KroQuant uses Kronecker-structured block transforms and LoRaQ calibration to enable efficient W4A4 post-training quantization for Diffusion Transformers.

Principles

• Activation outliers severely degrade 4-bit quantization quality in DiTs.
• Online transform inference cost is a binding constraint for PTQ in DiTs.
• Block-local Kronecker transforms can reduce parameters and accelerate quantization.

Method

KroQuant applies a learned Kronecker-structured invertible transform to 32-element activation blocks, then uses offline LoRaQ weight calibration to absorb residual quantization error.

In practice

• Implement Kronecker-structured transforms for efficient 4-bit DiT quantization.
• Combine block-local transforms with offline weight calibration for accuracy.

Topics

• KroQuant
• Post-Training Quantization
• Diffusion Transformers
• Kronecker Transforms
• W4A4 Quantization
• LoRaQ

Best for: Research Scientist, AI Engineer, AI Scientist, Machine Learning Engineer, Computer Vision Engineer

Related on AIssential

• FourTune: Towards Fully 4-Bit Efficient Post-Training for Diffusion Models — FourTune enables stable, efficient 4-bit post-training for large diffusion models by isolating outliers and optimizing low-bit computation.
• OrbitQuant: Data-Agnostic Quantization for Image and Video Diffusion Transformers — OrbitQuant enables data-agnostic quantization for Diffusion Transformers by quantizing in a normalized, rotated basis, overcoming activation shifts.
• KronQ: LLM Quantization via Kronecker-Factored Hessian — KronQ improves LLM post-training quantization by incorporating gradient covariance via Kronecker-factored Hessian approximation.
• Advanced MXFP4 Quantization: Combining Fine-Tuned Rotations with SmoothQuant for Near-Lossless Compression — Combining fine-tuned online rotations and SmoothQuant scales significantly improves MXFP4 quantization accuracy for language models.
• Bringing Nunchaku 4-bit Diffusion Inference to Diffusers — Nunchaku Lite integrates 4-bit W4A4 quantization into Diffusers, enabling faster, lower-VRAM diffusion model inference on consumer GPUs.
• AQ4SViT: An Automated Quantization Framework with Search Gating Policy for Compressing Spiking Vision Transformers — Automated quantization for SViTs uses performance proxies to quickly find optimal settings with accuracy-memory trade-offs.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Machine Learning.