Edit3DGS: Unified Framework for Dynamic Head Editing via 2D Instruction-Guided Diffusion and 3D Gaussian Splatting

2026-06-16 · Source: Computer Vision and Pattern Recognition · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Gaming & Interactive Media · Depth: Expert, quick

Summary

Edit3DGS is a unified framework designed for dynamic 3D head editing, integrating 2D instruction-guided diffusion with 3D Gaussian splatting. This approach overcomes limitations of previous methods by combining semantic controllability in the image domain with photorealistic, temporally consistent 3D representations. The framework processes an input video by masking editable facial regions and modifying them using a text-conditioned diffusion model, enabling fine-grained operations like expression transformation, attribute modification, and appearance refinement. Subsequently, the edited frames are aggregated through 3D Gaussian splatting to generate a coherent, high-fidelity avatar that maintains both identity and motion dynamics. Edit3DGS ensures temporal consistency via multi-view batch editing and lightweight inpainting strategies. This framework offers practical applications in virtual avatars, immersive communication, film production, and interactive media.

Key takeaway

For Computer Vision Engineers developing dynamic avatar systems or video editing tools, Edit3DGS offers a robust solution for high-fidelity 3D head manipulation. You can now achieve precise, instruction-guided facial edits in video while maintaining temporal consistency and identity. This framework simplifies complex tasks like expression transformation and attribute modification, potentially accelerating your development cycles for virtual communication or film production applications. Consider integrating this unified 2D diffusion and 3D splatting approach into your next-generation projects.

Key insights

Edit3DGS unifies 2D diffusion and 3D Gaussian splatting for dynamic, temporally consistent 3D head editing from video.

Principles

• Couple 2D semantic control with 3D photorealism.
• Preserve identity and motion dynamics in 3D avatars.
• Enforce temporal consistency via multi-view editing.

Method

Edit3DGS masks video facial regions, modifies them with text-conditioned diffusion, then aggregates edited frames via 3D Gaussian splatting, using multi-view batch editing and inpainting for consistency.

In practice

• Create virtual avatars with dynamic expressions.
• Enhance immersive communication experiences.
• Streamline film production for character edits.

Topics

• Dynamic 3D Head Editing
• 2D Instruction-Guided Diffusion
• 3D Gaussian Splatting
• Virtual Avatars
• Temporal Consistency
• Video Editing

Best for: Research Scientist, AI Scientist, Computer Vision Engineer, AI Engineer

Related on AIssential

• One-shot Compositional 3D Head Avatars with Deformable Hair — Decoupling hair from the face in 3D avatar generation significantly improves realism and dynamic behavior.
• Consistent-Inversion: Reverse Consistency Guidance for Structure-Preserving Visual Editing — Reverse consistency guidance corrects structural drift in diffusion-based image editing by checking trajectory reversibility.
• HorizonForge: Driving Scene Editing with Any Trajectories and Any Vehicles — HorizonForge enables photorealistic, controllable driving scene generation using 3D Gaussian Splats and meshes with video diffusion.
• Towards 3D-Aware Video Diffusion Models: Render-Free Human Motion Control with Mesh Tokenization — Video diffusion models can achieve 3D-awareness for human motion control by directly using compressed 3D human mesh tokens.
• Beyond Voxel 3D Editing: Learning from 3D Masks and Self-Constructed Data — BVE enables high-fidelity, text-guided 3D editing by integrating lightweight modules and a 3D masking strategy into a generative architecture.
• 🦜Geometry-Aware 4D Head🦜 👉 GeoDiff4D is a novel framework that reconstructs animatable... — GeoDiff4D reconstructs animatable 4D head avatars from a single image using geometry-aware diffusion.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computer Vision and Pattern Recognition.