Active Sampling for Ultra-Low-Bit-Rate Video Compression via Conditional Controlled Diffusion

2026-05-04 · Source: Computer Vision and Pattern Recognition · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Computer Vision · Depth: Expert, quick

Summary

ActDiff-VC is a novel diffusion-based video compression framework designed for ultra-low-bit-rate scenarios, leveraging conditional controlled diffusion models. The method segments videos into variable-length portions, transmitting keyframes only when necessary and summarizing temporal dynamics through compact tracked point trajectories. A conditional diffusion decoder then synthesizes the remaining frames based on these sparse signals, achieving perceptually realistic reconstructions under stringent rate constraints. ActDiff-VC incorporates content-adaptive keyframe selection and budget-aware sparse trajectory selection to ensure compact yet effective conditioning. Evaluations on the UVG and MCL-JCV benchmarks demonstrate that ActDiff-VC achieves up to 64.6% bitrate reduction at matched NIQE, improves KID by up to 64.6%, and FID by up to 37.7% at comparable bitrates against existing learned codecs, offering superior perceptual rate-distortion trade-offs.

Key takeaway

For research scientists developing next-generation video codecs, ActDiff-VC demonstrates a viable path to significantly reduce bitrates while maintaining perceptual quality. You should explore integrating conditional diffusion decoders with adaptive keyframe and trajectory selection mechanisms into your compression pipelines to achieve superior rate-distortion trade-offs in ultra-low-bit-rate applications.

Key insights

Conditional diffusion models can achieve ultra-low-bit-rate video compression through sparse, adaptive conditioning.

Principles

• Segment videos for adaptive processing.
• Transmit keyframes only when essential.
• Summarize temporal dynamics with point trajectories.

Method

ActDiff-VC partitions videos, transmits keyframes selectively, and uses tracked point trajectories as compact conditioning for a conditional diffusion decoder to synthesize frames, supported by content-adaptive keyframe and budget-aware trajectory selection.

In practice

• Apply content-adaptive keyframe selection.
• Utilize budget-aware sparse trajectory selection.

Topics

• Video Compression
• Diffusion Models
• Ultra-Low Bitrate
• ActDiff-VC
• Conditional Diffusion

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, Computer Vision Engineer

Related on AIssential

• Exploiting Semantic and Pixel Representations for Ultra-Low Bitrate Image Compression — Ultra-low bitrate image compression can achieve pixel-level fidelity by combining semantic and pixel representations in a diffusion model.
• Video Reconstruction using Diffusion-based Image-to-Video Generation with Trajectory Guidance — Trajectory-guided diffusion models can reconstruct missing video frames by integrating external sensor data.
• Physics-Informed Conditional Diffusion for Motion-Robust Retinal Temporal Laser Speckle Contrast Imaging — RetinaDiff uses physics-informed diffusion to reconstruct robust retinal blood flow from minimal speckle frames.
• Unlocking Diffusion Hierarchies: Adaptive Timestep Selection for Zero-Shot Segmentation — Diffusion models exhibit hierarchical semantic progression, enabling adaptive timestep selection for improved zero-shot segmentation.
• Adapting Diffusion Language Models for Lossless Pixel-Level Image Transmission — DDM-SSCC adapts diffusion language models for lossless pixel-level image transmission, achieving superior exact recovery via novel coding and denoising strategies.
• Stage-wise Distortion-Perception Traversal in Zero-shot Inverse Problems with Diffusion Models — A stage-wise diffusion model framework enables flexible distortion-perception tradeoff traversal in zero-shot inverse problems.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computer Vision and Pattern Recognition.