Clustered Codebook Quantization for 2D Gaussian-based Image Compression

2026-07-08 · Source: cs.CV updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Computer Graphics · Depth: Expert, medium

Summary

Cluster-Guided Vector Quantization (CGVQ) is a novel method for 2D Gaussian-based image compression, introduced in 2026, designed to enhance rate-distortion efficiency. It addresses the challenge of storing numerous floating-point parameters in Gaussian representations by partitioning Gaussian primitives into homogeneous groups using K-Means clustering prior to quantization. This approach enables the training of cluster-specific codebooks for position, rotation-scale, and color attributes. Extensive experiments demonstrate that CGVQ reduces the Bit-Per-Pixel (bpp) by 20% compared to the baseline GaussianImage (GI) model, while preserving on-par visual quality. Furthermore, CGVQ improves PSNR by 1.68 dB with the same 15K Gaussian primitives and achieves visual quality comparable to GI at 19K points, representing a 21% reduction in primitive count. The method, however, introduces a trade-off, as increasing cluster count from 1 to 16 decreases encoding FPS from 2.91e-2 to 1.9e-3 and decoding FPS from 133.3 to 33.3.

Key takeaway

For Machine Learning Engineers optimizing 2D Gaussian-based image compression, consider implementing cluster-guided vector quantization. Your approach should partition Gaussian primitives using K-Means on rotation, scale, and color features, then train cluster-specific codebooks. This method can reduce bpp by 20% and primitive count by 21% for comparable visual quality, but be mindful of the encoding/decoding speed trade-off as cluster numbers increase.

Key insights

Clustering Gaussian primitives before quantization significantly improves image compression rate-distortion performance.

Principles

• Narrower parameter distributions quantize more accurately.
• Localized codebooks improve reconstruction quality.
• K-Means clustering can optimize codebook quantization.

Method

Fit 2D Gaussian primitives to an image, then apply K-Means clustering on rotation, scale, and color features. Train three independent, cluster-specific codebooks for position (16-bit QAT), rotation-scale (UQ), and color (RQ), minimizing a joint reconstruction and VQ loss.

In practice

• Use K-Means to group similar image features.
• Apply multiple specialized codebooks per cluster.
• Consider performance trade-offs with cluster count.

Topics

• Image Compression
• 2D Gaussian Splatting
• Vector Quantization
• K-Means Clustering
• Rate-Distortion Optimization
• Codebook Quantization

Code references

• complight/Cluster_Guided_Vector_Quantization

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, Computer Vision Engineer

Related on AIssential

• Quantization robustness from dense representations of sparse functions in high-capacity kernel associative memory — KLR-trained Hopfield networks achieve quantization robustness via a "sparse function, dense representation" principle.
• Graph-based Clustering Revisited: A Relaxation of Kernel k-Means Perspective — LoRD and B-LoRD improve graph-based clustering by selectively relaxing kernel k-means constraints and using block diagonal regularization.
• Building a Masked Autoencoder (MAE) from Scratch: Teaching AI to See the Unseen — Masked Autoencoders learn robust visual representations by reconstructing highly masked images, leveraging image redundancy.
• Dynamic Cluster Data Sampling for Efficient and Long-Tail-Aware Vision-Language Pre-training — Dynamic cluster-based sampling efficiently trains VLMs, improving long-tail concept performance by balancing data distribution.
• How a 2021 Quantization Algorithm Quietly Outperforms Its 2026 Successor — Optimal analytical scaling in vector quantization significantly improves accuracy and efficiency over fixed scaling.
• Initialisation Determines the Basin: Efficient Codebook Optimisation for Extreme LLM Quantization — Codebook initialization is critical for extreme LLM quantization, especially at 2-bit precision.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CV updates on arXiv.org.