SlotMem: Character-Addressable Internal Memory for Narrative Long Video Generation

2026-07-17 · Source: Computer Vision and Pattern Recognition · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Computer Vision · Depth: Expert, quick

Summary

SlotMem is a novel character-addressable internal memory framework designed for multi-character narrative long video generation, addressing the challenge of maintaining recurring character identities across scene transitions and extended temporal gaps. Existing methods often struggle with global consistency due to retrieval cues not aligned with character identity or rely on coarse frame-level memory that entangles identity with incidental visual factors. SlotMem tackles this by employing a Character-Semantic Probe to localize character-relevant visual tokens from cross-attention responses. It then uses a Memory Encoder to compress DiT tokens into compact role-wise slot memory. A Memory Writer continuously updates each character's memory with new observations, while Character-Wise Cross-Attention injects the retrieved role memory exclusively into localized tokens of the corresponding character. Experiments on multiple narrative long video generation benchmarks demonstrate that SlotMem significantly improves long-range character consistency compared to existing baselines, all while preserving comparable video quality. The code for SlotMem is available on GitHub.

Key takeaway

For Machine Learning Engineers developing narrative long video generation models, SlotMem offers a robust solution for maintaining character identity. If your current models struggle with character consistency across long sequences or scene changes, you should consider integrating character-addressable memory. This approach, which localizes and selectively updates character-specific visual tokens, can significantly improve identity preservation without compromising video quality, streamlining your development of more coherent multi-character narratives.

Key insights

SlotMem uses character-addressable memory and selective attention to maintain consistent character identities in long video generation, outperforming prior methods.

Principles

• Character identity requires dedicated, continuously updated memory.
• Memory retrieval must be character-specific and localized.
• Compressing visual tokens into role-wise slots improves efficiency.

Method

SlotMem localizes character tokens via a Character-Semantic Probe, compresses DiT tokens into role-wise slot memory using a Memory Encoder, updates memory with a Memory Writer, and injects role memory via Character-Wise Cross-Attention.

In practice

• Implement character-specific memory for identity preservation.
• Use selective attention to inject memory into relevant tokens.
• Explore DiT token compression for efficient memory storage.

Topics

• Long Video Generation
• Character Consistency
• Internal Memory
• DiT Tokens
• Cross-Attention
• Computer Vision

Code references

• YilaiLiu-HKU/SlotMem

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, Computer Vision Engineer

Related on AIssential

• Your AI Hero Turns Into a Stranger Every Shot. The Fix Is a Document, Not a Better Prompt. — Character consistency in stateless AI pipelines requires a structured "character bible" and visual anchoring, not just better prompts.
• GroupVideo: Multi-Identity Customized Text-to-Video Generation — GroupVideo enables natural, consistent multi-identity text-to-video generation via multimodal identity alignment and spatial localization.
• LPM 1.0: Video-based Character Performance Model — LPM 1.0 resolves the "performance trilemma" for conversational AI by integrating data, multimodal conditioning, and streaming for real-time, expressive, and stable character video.
• EntityBench: Towards Entity-Consistent Long-Range Multi-Shot Video Generation — Maintaining entity consistency across long multi-shot video generation remains a significant challenge for current methods.
• LCG: Long-Context Consistent Image Generation with Sparse Relational Attention — LCG uses sparse attention and consistency constraints for scalable, coherent multi-image generation.
• MilliVid: Hierarchical Latents for Long-Range Consistency in Video Generation — Hierarchical latent token compression and coarse-to-fine rollout enable long-range consistency in video generation by managing detail levels efficiently.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computer Vision and Pattern Recognition.