InvSplat: Inverse Feed-Forward Scene Splatting

2026-07-02 · Source: Computer Vision and Pattern Recognition · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems, Emerging Technologies & Innovation · Depth: Expert, quick

Summary

InvSplat introduces a feed-forward multi-view reconstruction framework designed for inverse rendering, directly predicting a structured 3D Gaussian representation with intrinsic material attributes. Each Gaussian primitive is parameterized by mean, normal, opacity, rotation, scale, albedo, metallic, and roughness, creating a disentangled and physically grounded scene representation. The model integrates priors from a material estimation network with a multi-view 3D reconstruction backbone, enabling joint prediction of geometry and reflectance parameters in a single forward pass. Experiments on synthetic and real-world datasets demonstrate improved multi-view consistency compared to 2D baselines, accurate material recovery, and stable novel view rendering. This representation also supports physically-based relighting and more faithful modeling of view-dependent effects.

Key takeaway

For Computer Vision Engineers developing inverse rendering solutions, InvSplat offers a feed-forward approach that overcomes costly per-scene optimization and multi-view inconsistencies of 2D baselines. Its structured 3D Gaussian representation provides a robust foundation for accurate material recovery and physically-based relighting. Consider integrating this method to achieve more faithful modeling of view-dependent effects and stable novel view synthesis in your projects.

Key insights

InvSplat predicts 3D Gaussian representations with intrinsic material attributes for inverse rendering in a single forward pass.

Principles

• Gaussian primitives enable disentangled, physically grounded scene representation.
• Integrating material estimation priors improves joint geometry and reflectance prediction.

Method

InvSplat integrates priors from a material estimation network with a multi-view 3D reconstruction backbone to jointly predict geometry and reflectance parameters in a single forward pass.

In practice

• Supports physically-based relighting.
• Enables stable novel view rendering.
• Achieves accurate material recovery.

Topics

• Inverse Rendering
• 3D Gaussian Representation
• Multi-view Reconstruction
• Material Estimation
• Physically-Based Relighting
• Novel View Synthesis

Best for: Research Scientist, AI Scientist, Computer Vision Engineer

Related on AIssential

• Feed-Forward 3D Scene Modeling: A Problem-Driven Perspective — Feed-forward 3D reconstruction offers efficient, generalizable scene modeling by directly mapping 2D inputs to 3D representations.
• MoonSplat: Monocular Online Gaussian Splatting with Sim(3) Global Optimization — MoonSplat enables robust, efficient online 3D reconstruction from monocular video using globally optimized voxelized 3D Gaussian Splatting.
• WildSplatter: Feed-forward 3D Gaussian Splatting with Appearance Control from Unconstrained Images — WildSplatter enables fast, pose-free 3DGS from unconstrained images by disentangling geometry and appearance via learned embeddings.
• Extracting Neural Materials from Multi-view Images — NeuMatEx uses an LMRM and uncertainty-guided inverse rendering to extract complex neural materials from multi-view images.
• Less Gaussians, Texture More: 4K Feed-Forward Textured Splatting — LGTM enables 4K novel view synthesis by decoupling geometry from resolution using textured Gaussian primitives.
• Intrinsic decomposition and editing of 3D Gaussian splats — Intrinsic decomposition for 3D Gaussian splats enables texture editing independent of lighting across viewpoints.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Computer Vision and Pattern Recognition.