C-PTQ: Fisher-weighted Channel-wise Sensitivity for Post-training Quantization of MLLMs

2026-07-23 · Source: Computer Vision and Pattern Recognition · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, quick

Summary

C-PTQ is a novel post-training quantization (PTQ) method designed to address performance degradation in multimodal large language models (MLLMs) caused by outlier channels during quantization. MLLMs face significant memory and computational costs, making PTQ an attractive solution for compression and inference acceleration. Existing PTQ techniques often struggle because their channel-wise scaling (CWS) metrics, based on modality or token levels, fail to accurately capture the impact on task-specific loss. C-PTQ overcomes this by proposing a unified channel-wise approach that harmonizes task-specific loss perturbation with quantization error. It introduces a Fisher-weighted objective, serving as a tractable Hessian approximation, to seamlessly integrate task sensitivity into the scaling process. This method demonstrates leading performance without requiring auxiliary modules like LoRA, maintaining high efficiency. Its effectiveness has been demonstrated through experiments on MLLMs such as Qwen2.5VL, InternVL2, and LLaVA-OV across 8 benchmarks, covering both weight-only and weight-activation quantization settings.

Key takeaway

For machine learning engineers deploying multimodal large language models (MLLMs) where memory and computational costs are critical, C-PTQ provides a robust solution. You can achieve leading post-training quantization performance, mitigating accuracy degradation from outlier channels. This method eliminates the need for auxiliary modules like LoRA, simplifying your deployment pipeline and maintaining high efficiency. Consider integrating C-PTQ to optimize MLLM inference on resource-constrained hardware.

Key insights

C-PTQ uses Fisher-weighted channel-wise sensitivity to harmonize task-specific loss and quantization error for MLLM post-training quantization.

Principles

• Outlier channels significantly impair MLLM quantization accuracy.
• Task-specific loss perturbation is crucial for effective channel scaling.
• Second-order derivatives can approximate task sensitivity for quantization.

Method

C-PTQ designs a Fisher-weighted objective as a tractable Hessian approximation. This objective seamlessly injects task sensitivity into the channel-wise scaling process, harmonizing task-specific loss perturbation and quantization error.

In practice

• Apply C-PTQ for efficient MLLM deployment.
• Quantize Qwen2.5VL, InternVL2, LLaVA-OV models.
• Achieve leading efficiency without LoRA modules.

Topics

• Multimodal LLMs
• Post-training Quantization
• Channel-wise Sensitivity
• Model Compression
• Inference Acceleration
• Qwen2.5VL

Best for: NLP Engineer, Computer Vision Engineer, Research Scientist, AI Scientist, Machine Learning Engineer, AI Engineer

Related on AIssential

• Minimizing the Hidden Cost of Scales: Graph-Guided Ultra-Low-Bit Quantization for Large Language Models — SAGE-PTQ uses graph-guided dual-mode quantization to achieve ultra-low-bit LLM inference with minimal scaling overhead.
• Quantizing Time-Series Models As Dynamical Systems: Trajectory-Based Quantization Sensitivity Score — Reframing post-training quantization as a dynamical system allows a priori error sensitivity estimation for robust low-precision deployment.
• Channel-Wise Mixed-Precision Quantization for Large Language Models — CMPQ adaptively quantizes LLMs channel-wise using mixed-precision and outlier protection for efficient edge deployment.
• TWLA: Achieving Ternary Weights and Low-Bit Activations for LLMs via Post-Training Quantization — TWLA enables efficient LLM inference by jointly quantizing weights to 1.58-bit and activations to 4-bit via a three-module PTQ framework.
• SAB-LVLM: Significance-Aware Binarization for Large Vision-Language Models — SAB-LVLM improves LVLM binarization by adaptively weighting parameters based on their cross-modal significance, achieving superior compression with minimal performance loss.
• KronQ: LLM Quantization via Kronecker-Factored Hessian — KronQ improves LLM post-training quantization by incorporating gradient covariance via Kronecker-factored Hessian approximation.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computer Vision and Pattern Recognition.