OrbitQuant: Data-Agnostic Quantization for Image and Video Diffusion Transformers

2026-07-02 · Source: Artificial Intelligence · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Emerging Technologies & Innovation · Depth: Expert, quick

Summary

OrbitQuant is a data-agnostic weight-activation quantizer designed for Diffusion Transformers (DiTs), which are known for expensive inference due to multi-step sampling and increasing parameter counts. Traditional post-training quantization (PTQ) methods struggle with DiT activation shifts across timesteps, prompts, and guidance branches, often requiring re-fitting calibration data. OrbitQuant addresses this by quantizing in a normalized, rotated basis. It employs a randomized permuted block-Hadamard (RPBH) rotation to concentrate each coordinate around a fixed marginal, allowing a single Lloyd-Max codebook to serve all timesteps, prompts, and layers. The method extends to weight rows offline, absorbing the rotation into the weights, so only a forward rotation on activations is needed at runtime. This approach transfers seamlessly from image to video generation without per-modality tuning, achieving state-of-the-art PTQ at low-bit settings across models like FLUX.1, Z-Image-Turbo, Wan 2.1, and CogVideoX, and pushing image DiT PTQ to W2A4 with usable quality.

Key takeaway

For Machine Learning Engineers optimizing Diffusion Transformer inference costs, OrbitQuant provides a significant advancement. This data-agnostic quantization method eliminates the need for re-fitting calibration data across varying timesteps, prompts, or modalities, simplifying your workflow. By quantizing in a normalized, rotated basis, it achieves state-of-the-art low-bit performance, including W2A4 for image DiTs. You should evaluate OrbitQuant to reduce memory and computational demands for your image and video generation models like FLUX.1 or CogVideoX.

Key insights

OrbitQuant enables data-agnostic quantization for Diffusion Transformers by quantizing in a normalized, rotated basis, overcoming activation shifts.

Principles

• Quantize in a normalized, rotated basis.
• RPBH rotation stabilizes activation distributions.
• Absorb rotation into weights offline.

Method

OrbitQuant quantizes in a normalized, rotated basis using a randomized permuted block-Hadamard (RPBH) rotation. This concentrates coordinates, enabling a single Lloyd-Max codebook. Weight rotations are absorbed offline, leaving only activation rotation at runtime.

In practice

• Apply to image and video DiTs.
• Achieve W2A4 PTQ for image DiTs.
• Improve inference for FLUX.1, Z-Image-Turbo.

Topics

• Diffusion Transformers
• Post-Training Quantization
• Data-Agnostic Quantization
• Image Generation
• Video Generation
• Model Compression

Best for: Research Scientist, AI Engineer, AI Scientist, Machine Learning Engineer, Computer Vision Engineer

Related on AIssential

• Advanced MXFP4 Quantization: Combining Fine-Tuned Rotations with SmoothQuant for Near-Lossless Compression — Combining fine-tuned online rotations and SmoothQuant scales significantly improves MXFP4 quantization accuracy for language models.
• Sol Video Inference Engine: Agent-Native Full-Stack Acceleration Framework for Efficient Video Generation — The Sol Video Inference Engine uses an agentic framework to dynamically optimize video diffusion model acceleration for instance-specific deployments.
• STaR-Quant: State-Time Consistent Post-Training Quantization for Diffusion Large Language Models — STaR-Quant improves Diffusion LLM efficiency by addressing state-dependent activation disparity and temporal error accumulation in low-bit quantization.
• ActQuant: Sub-4-bit Action-Guided Quantization for Vision-Language-Action Models — ActQuant enables sub-4-bit quantization for VLA models by action-guided mixed-precision allocation and efficient C/C++ deployment.
• CoReDiT: Spatial Coherence-Guided Token Pruning and Reconstruction for Efficient Diffusion Transformers — CoReDiT efficiently prunes Diffusion Transformers by leveraging spatial coherence for token selection and reconstruction, significantly reducing computational cost.
• A Note on TurboQuant and the Earlier DRIVE/EDEN Line of Work — TurboQuant is a suboptimal variant of the EDEN quantization scheme, particularly due to its fixed scale parameter.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Artificial Intelligence.