TeCoNeRV: Leveraging Temporal Coherence for Compressible Neural Representations for Videos

2026-02-18 · Source: Computer Vision and Pattern Recognition · Field: Technology & Digital — Artificial Intelligence & Machine Learning · Depth: Expert, quick

Summary

TeCoNeRV is a novel method that significantly improves video compression using Implicit Neural Representations (INRs) by addressing the limitations of existing hypernetwork-based approaches. It decomposes weight prediction spatially and temporally, processing short video segments into patch tubelets, which reduces pretraining memory overhead by 20x. TeCoNeRV employs a residual-based storage scheme to capture only differences between consecutive segment representations, thereby reducing bitstream size. Additionally, it incorporates a temporal coherence regularization framework that correlates weight space changes with video content. This approach achieves substantial PSNR improvements of 2.47dB at 480p and 5.35dB at 720p on the UVG dataset, with 36% lower bitrates and 1.5-3x faster encoding speeds. TeCoNeRV is the first hypernetwork method to demonstrate results at 480p, 720p, and 1080p across UVG, HEVC, and MCL-JCV datasets.

Key takeaway

For research scientists developing video compression algorithms, TeCoNeRV offers a pathway to overcome memory and quality limitations of hypernetwork-based INRs. You should investigate its spatial-temporal decomposition and residual storage scheme to achieve higher PSNR, lower bitrates, and faster encoding speeds for high-resolution video applications, potentially enabling new practical deployments.

Key insights

TeCoNeRV enhances video compression via INRs by leveraging temporal coherence and spatial-temporal decomposition.

Principles

• Decompose complex tasks for efficiency
• Store residuals for data reduction
• Correlate weight changes with content

Method

TeCoNeRV spatially and temporally decomposes weight prediction into patch tubelets, uses a residual-based storage for segment differences, and applies temporal coherence regularization.

In practice

• Reduce memory for high-res video INRs
• Achieve 36% lower video bitrates
• Speed up video encoding 1.5-3x

Topics

• Implicit Neural Representations
• Video Compression
• Hypernetworks
• Temporal Coherence
• Computer Vision

Best for: Research Scientist, AI Researcher, AI Scientist, Computer Vision Engineer

Related on AIssential

• Fre-Res: Frequency-Residual Video Token Compression for Efficient Video MLLMs — Fre-Res efficiently compresses video for MLLMs by separating spatial anchors from compact temporal-frequency residuals.
• Fast Amortized Fitting of Scientific Signals Across Time and Ensembles via Transferable Neural Fields — Transferable neural fields enable efficient, scalable representation of high-dimensional scientific data by reusing learned features.
• You Don't Need Strong Assumptions: Visual Representation Learning via Temporal Differences — Temporal Difference in Vision (TDV) learns visual representations from video by predicting future frames from past motion, eliminating strong inductive biases.
• Residual Modeling for High-Fidelity Learned Compression of Scientific Data — Tailored residual representations preserve learned compression advantages in high-fidelity scientific data scenarios.
• MoRe: Motion-aware Feed-forward 4D Reconstruction Transformer — MoRe efficiently reconstructs dynamic 4D scenes from monocular video by disentangling motion from static structure.
• Revisiting Token Compression for Accelerating ViT-based Sparse Multi-View 3D Object Detectors — Dynamic patch size adjustment and semantic preservation accelerate ViT-based 3D object detection without accuracy loss.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computer Vision and Pattern Recognition.