GenEraser: Generalizable Video Object Removal via Balanced Text-Mask Guidance and Decoupled Locator-Preserver

2026-05-28 · Source: Computer Vision and Pattern Recognition · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Computer Vision · Depth: Expert, quick

Summary

GenEraser is a novel framework designed for generalized and high-fidelity video object and effect removal, addressing challenges like complex spatiotemporal ambiguities and the failure of spatial masks to capture weakly correlated effects. It tackles the fundamental optimization conflict between high-level semantic generalization and precise pixel-level background preservation. The system introduces a Multi-Conditional Mixture-of-Experts (MC-MoE) paired with Bipartite Text guidance to fully exploit Diffusion Transformers for identifying complex effects. A Learnable Deep "CFG" Fusion (LD-CFG) mechanism adaptively balances mask and textual conditions. Furthermore, a Decoupled Expert Architecture, comprising a Locator and a Preserver, mitigates the inherent trade-off. GenEraser significantly outperforms state-of-the-art methods, achieving 2.16 dB on the ROSE Benchmark and 1.44 dB on VOR-Eval, demonstrating robust generalization in open-world scenarios.

Key takeaway

For Computer Vision Engineers developing advanced video editing tools, GenEraser offers a robust solution for complex object and effect removal. Its novel architecture, combining multimodal guidance and a decoupled expert system, directly addresses the trade-off between semantic generalization and pixel-level background preservation. You should consider integrating similar balanced text-mask guidance and decoupled processing to achieve superior fidelity and generalization in your video processing pipelines.

Key insights

GenEraser enhances video object removal by integrating multimodal guidance and a decoupled architecture to manage complex effects and preserve backgrounds.

Principles

• Spatial masks alone miss weakly correlated effects.
• Textual guidance improves complex effect identification.
• Semantic generalization conflicts with pixel preservation.

Method

GenEraser employs MC-MoE with Bipartite Text guidance, LD-CFG for condition balancing, and a Decoupled Expert Architecture (Locator, Preserver) to resolve semantic-pixel trade-offs.

In practice

• Removes objects and associated physical effects.
• Maintains fidelity in out-of-domain videos.
• Improves generalization for complex scenarios.

Topics

• Video Object Removal
• Diffusion Transformers
• Multi-Conditional Mixture-of-Experts
• Text-Mask Guidance
• Decoupled Architecture
• Computer Vision

Best for: Research Scientist, AI Scientist, Computer Vision Engineer

Related on AIssential

• RefDecoder: Enhancing Visual Generation with Conditional Video Decoding — Conditional video decoders improve detail preservation and consistency in visual generation.
• CreatiParser: Generative Image Parsing of Raster Graphic Designs into Editable Layers — CreatiParser decomposes raster graphic designs into editable layers using a hybrid generative framework for enhanced editing.
• Beyond Voxel 3D Editing: Learning from 3D Masks and Self-Constructed Data — BVE enables high-fidelity, text-guided 3D editing by integrating lightweight modules and a 3D masking strategy into a generative architecture.
• Building Generative Image & Video models at Scale - Sander Dieleman (Veo and Nano Banana) — Diffusion models generate high-quality audiovisual data through iterative denoising of learned compressed latent representations.
• Introducing V-RAG: revolutionizing AI-powered video production with Retrieval Augmented Generation — V-RAG enhances AI video generation by integrating image retrieval, offering customization and accuracy without model retraining.
• Seen-to-Scene: Keep the Seen, Generate the Unseen for Video Outpainting — Seen-to-Scene unifies propagation and generation for video outpainting, improving temporal coherence and realism.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computer Vision and Pattern Recognition.