Bigger or Cheaper? Scale and Quantization Effects on Uncertainty Signals in Vision-Language Models Under Image Degradation

2026-07-27 · Source: Computation and Language · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Computer Vision, Natural Language Processing · Depth: Expert, quick

Summary

A study on Vision-Language Models (VLMs) investigates the impact of model scale and 4-bit quantization on uncertainty signals, particularly within the Qwen2-VL family, when processing images with six types of photographic degradation at three severities. Across 5,700 predictions, researchers found that increasing model scale from 2B to 7B parameters sharply improves the model's internal uncertainty signal, with mean error-detection AUROC rising from 0.80 to 0.98. However, verbalized confidence remains weak (mean AUROC 0.61 to 0.69), indicating a widening gap between internal knowledge and stated confidence as models grow. While 4-bit quantization incurs only a minor accuracy penalty of -1.6 points, it significantly degrades the internal confidence signal (AUROC drops from 0.95 to 0.80) and reduces the verbalized-confidence parse rate from 99% to 64%. For a fixed memory budget, the research recommends preferring a larger quantized model, such as 7B-4bit, over a smaller full-precision one, as it achieves both superior accuracy and the best internal uncertainty signal (AUROC 0.98) among configurations tested.

Key takeaway

For AI Engineers deploying Vision-Language Models on consumer hardware with fixed memory, you should prioritize larger, 4-bit quantized models over smaller, full-precision alternatives. This approach yields superior accuracy and a more reliable internal uncertainty signal, crucial for selective prediction. Be aware that verbalized confidence may not reflect the model's true internal certainty, so rely on metrics like error-detection AUROC for robust decision-making.

Key insights

Model scale improves internal VLM uncertainty signals more than verbalized confidence, while 4-bit quantization degrades confidence significantly.

Principles

• Larger VLMs improve internal uncertainty signals.
• 4-bit quantization impacts confidence more than accuracy.
• Error-detection AUROC is key for VLM uncertainty.

Method

The study measured two confidence signals (natural language and mean token probability) in Qwen2-VL models under image degradation, comparing scale and 4-bit quantization effects.

In practice

• Prefer larger quantized VLMs for fixed memory.
• Monitor internal uncertainty signals (AUROC).
• Expect verbalized confidence to lag internal knowledge.

Topics

• Vision-Language Models
• Model Quantization
• Uncertainty Quantification
• Image Degradation
• Qwen2-VL
• Error-Detection AUROC

Best for: Computer Vision Engineer, Research Scientist, AI Scientist, Machine Learning Engineer, AI Engineer

Related on AIssential

• From 32 Bits to 1.58: The Illustrated Guide to LLM Quantization — LLM quantization significantly reduces memory and accelerates inference while maintaining intelligence across various bit precisions.
• Uncertainty Quantification for Multimodal Large Language Models with Incoherence-adjusted Semantic Volume — UMPIRE quantifies MLLM uncertainty by measuring semantic volume and response incoherence without additional training or external tools.
• Diagnosing Visual Ignorance in Vision-Language Models — VLMs' visual ignorance results from internal routing failures and benchmarks that reward language-prior reliance.
• Small Vision-Language Models Know When They Are Wrong But Cannot Say So: A Two-Model Study of Stated versus Internal Confidence Under Realistic Image Degradation — Small VLMs possess internal self-knowledge about errors that their verbalized confidence does not express.
• Comparing Qwen3-VL AI Models for OCR Task — Quantized larger Qwen3-VL models can outperform smaller, unquantized versions in both speed and accuracy for complex OCR.
• When Thinking Hurts: Epistemic Signals in the Reasoning Chains of Visual Language Models — Reasoning chain entropy is a more reliable uncertainty signal than answer entropy in thinking-mode VLMs.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computation and Language.