TivTok: Broadcasting Time-Invariant Tokens for Scalable Video Tokenization

2026-06-16 · Source: Computer Vision and Pattern Recognition · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Computer Vision · Depth: Expert, quick

Summary

TivTok (Time-Invariant Tokenizer) is a novel video tokenization method designed to enhance scalability in video generation by making persistent content reusable across time. Unlike existing tokenizers that repeatedly represent static backgrounds or consistent object appearances, TivTok factorizes video clips into Time-Invariant (TIV) tokens, encoding shared information, and Time-Variant (TV) tokens, capturing frame-specific residuals. This factorization is achieved through Scope-Induced Factorization (SIF), which assigns distinct attention scopes. Invariant Broadcasting (IB) then reuses TIV tokens for parallel reconstruction and long-video tokenization. Experiments demonstrate TivTok achieves an rFVD of 12.65 on the 16x256x256 benchmark, improves compression efficiency by 2.91x for 128-frame videos, and uses only 1.1% of tokens compared to downsample-based tokenizers.

Key takeaway

For Machine Learning Engineers developing video generation models, TivTok offers a significant advancement in managing computational costs and enabling longer video sequences. By drastically reducing the number of tokens required—using only 1.1% compared to traditional methods and improving compression by 2.91x—you can achieve greater scalability. Consider integrating TivTok's reuse-aware tokenization to build more efficient and capable video generation systems.

Key insights

TivTok reuses persistent video content information across frames to reduce token count and computational cost for scalable video generation.

Principles

• Factorize video content into time-invariant and time-variant components.
• Assign distinct attention scopes for efficient token processing.
• Broadcast invariant tokens for parallel reconstruction.

Method

TivTok uses Scope-Induced Factorization (SIF) to separate Time-Invariant (TIV) and Time-Variant (TV) tokens. TIV tokens attend to full clips; TV tokens access their frame and TIVs. Invariant Broadcasting (IB) reuses TIVs for reconstruction.

In practice

• Scalable video generation.
• Long-video tokenization.
• Reduced computational cost.

Topics

• Video Tokenization
• TivTok
• Time-Invariant Tokens
• Video Generation
• Compression Efficiency
• Computer Vision

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, Computer Vision Engineer

Related on AIssential

• EVATok: Adaptive Length Video Tokenization for Efficient Visual Autoregressive Generation — EVATok adaptively tokenizes video segments to optimize quality-cost trade-offs in autoregressive video generation.
• HYDRA-X: Native Unified Multimodal Models with Holistic Visual Tokenizers — HYDRA-X unifies image and video tokenization in a single ViT, improving multimodal model efficiency and semantic awareness.
• Don't Pause: Streaming Video-Language Synchrony for Online Video Understanding — Streaming Video-Language Synchrony enables real-time, stutter-free video-language interaction by interleaving perception and generation at frame-token granularity.
• EarlyTom: Early Token Compression Completes Fast Video Understanding — Early, in-encoder token compression significantly boosts Video-LLM efficiency by optimizing vision encoding.
• Balancing Image Compression and Generation with Bootstrapped Tokenization — SelfBootTok decomposes image tokens into global and local groups, predicting local details from global tokens for efficient generation and scalable training.
• AdaCodec: A Predictive Visual Code for Video MLLMs — AdaCodec uses a predictive visual code to reduce redundancy in video MLLMs, improving efficiency and performance.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computer Vision and Pattern Recognition.