Group-of-Latents: Perceptual Video Compression at Extreme Bitrates via Masked Latent Generative Modeling

2026-07-21 · Source: Computer Vision and Pattern Recognition · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Emerging Technologies & Innovation · Depth: Expert, quick

Summary

Group-of-Latents (GoL) is a novel generative framework for perceptual video compression, specifically designed for extremely low bitrates, such as below 0.005 bpp. This method addresses the underexplored challenge of optimizing perceptual quality under severe coding resource constraints, a limitation of traditional transformation and quantization algorithms. GoL incorporates pre-trained Diffusion Transformer (DiT) priors and introduces a flexible strategy within a causal tokenizer's latent space, partitioning the latent stream into intra $I$-latents and inter $P$-latents. A Deep Compression Module (I-DCM) encodes key $I$-latents to establish perceptual anchors with minimal overhead. Subsequently, a DiT-based Unified Latent Denoising Module (U-LDM) refines intra-frame textures and synthesizes $P$-latents from noise, reconstructing temporal dynamics without additional bitrate cost. Experiments show GoL achieves high perceptual fidelity, rich spatial details, and robust temporal consistency at these extreme bitrates.

Key takeaway

For Computer Vision Engineers developing systems requiring extreme-low-bitrate video compression, Group-of-Latents (GoL) offers a significant advancement. If your applications demand high perceptual quality below 0.005 bpp, you should explore this generative framework. It enables robust temporal consistency and rich spatial details, fundamentally changing how you might approach video encoding for bandwidth-constrained environments or efficient storage.

Key insights

GoL uses DiT priors for extreme-low-bitrate perceptual video compression, partitioning latents for efficient reconstruction.

Principles

• Optimizing perceptual quality is key at extreme bitrates.
• Latent space partitioning improves compression efficiency.
• Generative models can synthesize temporal dynamics.

Method

GoL partitions latent streams into intra $I$-latents and inter $P$-latents. An I-DCM encodes $I$-latents, while a DiT-based U-LDM refines textures and synthesizes $P$-latents from noise for temporal reconstruction.

In practice

• Achieve high perceptual quality below 0.005 bpp.
• Use DiT priors for generative video compression.
• Separate intra and inter latent processing.

Topics

• Video Compression
• Perceptual Quality
• Diffusion Transformer
• Latent Generative Models
• Extreme Bitrates
• Deep Compression

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, Computer Vision Engineer

Related on AIssential

• Group-of-Latents: Perceptual Video Compression at Extreme Bitrates via Masked Latent Generative Modeling — Decoupling spatial and temporal video latents with generative priors enables extreme-low bitrate perceptual compression.
• Hybrid Latents -- Geometry-Appearance-Aware Surfel Splatting — Hybrid Latents improve 3D scene reconstruction by separating geometry and appearance using a novel Gaussian-hash-grid representation.
• Exploiting Semantic and Pixel Representations for Ultra-Low Bitrate Image Compression — Ultra-low bitrate image compression can achieve pixel-level fidelity by combining semantic and pixel representations in a diffusion model.
• Frozen Forecasting: A Unified Evaluation — Perceptual ability in frozen video models strongly correlates with their generalist forecasting performance over short time horizons.
• Addressing Detail Bottlenecks in Latent Diffusion for RGB-to-SWIR Image Translation — Latent Diffusion Models can overcome detail loss in image translation by enhancing autoencoder and conditioning pathways with learned, high-resolution feature injection.
• Latent Spatial Memory for Video World Models — Latent spatial memory in diffusion models significantly boosts video generation speed and reduces memory by avoiding pixel-space reconstruction.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computer Vision and Pattern Recognition.