ReQAT: Achieving Full-Precision Reasoning Accuracy with 4-bit Floating-Point Quantization-Aware Training

2026-06-14 · Source: Machine Learning · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, quick

Summary

ReQAT is a novel 4-bit floating-point quantization-aware training framework designed to overcome severe reasoning degradation in Large Reasoning Models (LRMs) when fully quantizing weights, activations, and KV caches (W4A4KV4). Existing quantization methods fail to recover accuracy in these scenarios, particularly impacting low-entropy tokens like digits and operators, where quantization noise inflates sampling errors. ReQAT addresses this with three components: Trace-Aligned QAT (TAQ) focuses updates on critical low-entropy decisions by revisiting identical reasoning traces; Selective Entropy Minimization (SEM) reinforces confidence at these positions; and Q-FIT provides a quantization-friendly initialization, calibrating RoPE-consistent KV cache transformations. This framework not only recovers but surpasses BF16 fine-tuning accuracy under the same training budget, achieving up to 3.9x throughput speedup on NVIDIA DGX Spark and 3.1x on B200.

Key takeaway

For Machine Learning Engineers deploying Large Reasoning Models, ReQAT offers a path to significantly reduce inference costs and KV cache footprints without sacrificing accuracy. You should investigate integrating 4-bit floating-point quantization-aware training, specifically focusing on techniques that address low-entropy token precision. This approach allows you to achieve up to 3.9x throughput speedup while potentially surpassing BF16 fine-tuning accuracy, making efficient LRM deployment feasible for resource-constrained environments.

Key insights

ReQAT enables 4-bit quantization for LRMs, recovering full-precision reasoning accuracy by targeting low-entropy token sensitivity.

Principles

• Low-entropy tokens are critical for reasoning accuracy.
• Quantization noise amplifies errors in symbolic commitments.
• Targeted QAT can recover reasoning performance.

Method

ReQAT employs Trace-Aligned QAT (TAQ) for critical decision updates, Selective Entropy Minimization (SEM) for confidence reinforcement, and Q-FIT for quantization-friendly initialization and KV cache calibration.

In practice

• Deploy LRMs with W4A4KV4 for efficiency.
• Focus QAT on low-entropy token precision.
• Use RoPE-consistent KV cache transformations.

Topics

• Quantization-Aware Training
• Large Reasoning Models
• FP4 Quantization
• KV Cache Optimization
• Model Inference Speedup
• Low-Entropy Tokens

Best for: AI Engineer, NLP Engineer, AI Architect, AI Scientist, Machine Learning Engineer, MLOps Engineer

Related on AIssential

• StatQAT: Statistical Quantizer Optimization for Deep Networks — A statistical framework optimizes uniform and floating-point quantization parameters for deep neural networks during training.
• Extreme Low-Bit Inference in Reasoning Models: Failure Modes and Targeted Recovery — Aggressive low-bit quantization in LRMs causes specific reasoning failures, recoverable with targeted high-precision planning and loop detection.
• KVarN: new KV-cache quant from Huawei. 3–5× KV cache compression with actual speed-up instead of slow-down, and unlike TurboQuant it holds up on reasoning (Apache 2.0, vLLM single flag) — KVarN offers superior KV-cache compression and throughput without sacrificing reasoning quality, unlike prior methods.
• Google Shrank Gemma 4 by 72% and Unsloth Fixed the 4-Bit Bug Nobody Else Caught on One 4090, and… — Google's Gemma 4 QAT and Unsloth's fix enable 4-bit quantized models to achieve near full-precision performance on consumer hardware.
• AAAC: Activation-Aware Adaptive Codebooks for 4-bit LLM Weight Quantization — Adaptive codebooks, learned via activation-weighted k-means, significantly improve 4-bit LLM quantization without high computational cost.
• Google's TurboQuant Crashed the AI Chip Market — Google's Turbo Quant algorithm offers 6x memory reduction and 8x speedup for LLM inference with zero accuracy loss.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Machine Learning.