FourTune: Towards Fully 4-Bit Efficient Post-Training for Diffusion Models

2026-01-21 · Source: cs.CV updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, extended

Summary

FourTune is an efficient post-training framework for large diffusion models, designed to overcome prohibitive memory footprints and slow training speeds. It implements an end-to-end W4A4G4 paradigm, quantizing weights, activations, and gradients to 4-bit precision. The framework integrates a triple-branch hybrid pipeline, augmenting the standard LoRA architecture with a frozen numerical stabilizer to isolate quantization-sensitive outliers, ensuring stable training. Additionally, FourTune utilizes hardware-efficient block-wise quantization and customized fused kernels to support efficient quantized backpropagation and reduce memory bandwidth overhead. Across customization, reinforcement learning, and distillation tasks, FourTune matches the quality of full-precision fine-tuning. On FLUX.1-dev (12B), it reduces memory overhead by 2.25× and increases end-to-end training throughput by 2.27× compared to BF16 LoRA.

Key takeaway

For Machine Learning Engineers adapting large diffusion models, FourTune offers a significant efficiency improvement. If you are constrained by GPU memory or training speed, adopting FourTune's W4A4G4 framework can reduce memory overhead by 2.25× and accelerate training by 2.27× compared to BF16 LoRA, without sacrificing generation quality. Consider integrating its triple-branch pipeline and kernel fusion for practical, high-performance post-training.

Key insights

FourTune enables stable, efficient 4-bit post-training for large diffusion models by isolating outliers and optimizing low-bit computation.

Principles

• Isolate quantization-sensitive outliers for numerical stability.
• Quantize weights, activations, and gradients (W4A4G4).
• Employ block-wise quantization for efficient 4-bit backpropagation.

Method

FourTune uses a triple-branch hybrid pipeline with a frozen 4-bit backbone, a frozen full-precision stabilizer for outliers, and a trainable LoRA branch. It employs block-wise quantization for 4-bit backward pass and kernel fusion for LoRA and MLP modules.

In practice

• Apply W4A4G4 for diffusion model fine-tuning.
• Implement a numerical stabilizer to prevent gradient explosion.
• Fuse LoRA and MLP kernels to boost training throughput.

Topics

• Diffusion Models
• 4-bit Quantization
• Post-Training
• LoRA
• Parameter-Efficient Fine-Tuning
• Kernel Fusion

Code references

• black-forest-labs/flux
• chaofengc/IQA-PyTorch
• discus0434/aesthetic-predictor-v2-5

Best for: Research Scientist, AI Engineer, Computer Vision Engineer, AI Scientist, Machine Learning Engineer, AI Hardware Engineer

Related on AIssential

• Collaborative Few-Step Distillation and Low-Bit Quantization for Wan2.2 Dual-Expert Video Diffusion Models — Co-designing distillation and low-bit quantization compresses video diffusion models, surpassing full-precision at fewer steps.
• Google Shrank Gemma 4 by 72% and Unsloth Fixed the 4-Bit Bug Nobody Else Caught on One 4090, and… — Google's Gemma 4 QAT and Unsloth's fix enable 4-bit quantized models to achieve near full-precision performance on consumer hardware.
• Post-Training Memory Reduction Techniques for Model Inference — Post-training memory reduction for inference optimizes VRAM, latency, and cost without retraining.
• STaR-Quant: State-Time Consistent Post-Training Quantization for Diffusion Large Language Models — STaR-Quant improves Diffusion LLM efficiency by addressing state-dependent activation disparity and temporal error accumulation in low-bit quantization.
• ScaleSweep: Accurate NVFP4 Post-Training Quantization of LLMs via Block Scale Initialization — ScaleSweep optimizes NVFP4 quantization by sweeping block scale candidates within derived theoretical bounds to minimize reconstruction error.
• Synthetic Benchmarks Overstate Forward-Forward Scaling: Real-Data Limits of Layer-Local Training — Forward-Forward learning's real-data scaling and memory efficiency are significantly overstated by synthetic benchmarks, revealing limitations compared to backpropagation.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CV updates on arXiv.org.