Neural Harmonic Textures for High-Quality Primitive Based Neural Reconstruction

2026-07-11 · Source: cs.AI updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems, Emerging Technologies & Innovation · Depth: Expert, extended

Summary

Neural Harmonic Textures (NHT) is a novel neural representation that enhances primitive-based 3D reconstruction, yielding high-quality results in real-time novel view synthesis. It addresses the limited expressivity of individual primitives in methods like 3D Gaussian Splatting by anchoring latent feature vectors on a virtual scaffold surrounding each primitive. These features are interpolated at ray intersection points, then processed with periodic activations (sine and cosine) to transform alpha blending into a weighted sum of harmonic components. A small neural network decodes the resulting signal in a single deferred pass, significantly reducing computational cost. NHT integrates seamlessly into existing pipelines such as 3DGUT, Triangle Splatting, and 2DGS, and demonstrates generality in applications like 2D image fitting and semantic reconstruction. It consistently outperforms baselines, including 3DGS-MCMC and Instant NGP, while maintaining high inference performance (140+ FPS).

Key takeaway

For Machine Learning Engineers developing real-time 3D reconstruction or novel view synthesis systems, Neural Harmonic Textures offer a significant leap in detail fidelity without sacrificing performance. You should consider integrating this primitive-based neural representation into your pipelines to achieve superior visual quality, especially for high-frequency details and view-dependent effects. Its adaptability to motion and scene editing, combined with efficient deferred decoding, makes it a robust choice for scalable and editable scene representations.

Key insights

Neural Harmonic Textures enhance primitive expressivity by encoding interpolated features with periodic activations for efficient, high-fidelity reconstruction.

Principles

• Anchor features to primitives for Lagrangian adaptability.
• Periodic activations decompose signals into harmonics.
• Deferred decoding reduces neural network evaluations.

Method

Latent feature vectors are anchored to a virtual scaffold (e.g., tetrahedra) around primitives, barycentrically interpolated at ray intersections, then harmonically encoded with sine/cosine functions, and finally decoded by a shallow MLP in a single deferred pass.

In practice

• Integrate into 3DGUT, Triangle Splatting, 2DGS pipelines.
• Apply to semantic field reconstruction with expanded MLP.
• Use for high-resolution 2D image fitting with C1 interpolation.

Topics

• Neural Reconstruction
• 3D Gaussian Splatting
• Novel View Synthesis
• Harmonic Textures
• Deferred Shading
• Primitive-based Methods
• Real-time Rendering

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, Computer Vision Engineer

Related on AIssential

• Recurrent Sinusoidal INRs for Efficient High-Fidelity Representation — Sinusoidal recurrence iteratively enriches harmonic spectral support in implicit neural representations.
• Neural Texture Compression using Hypernetworks — A single hypernetwork can generate both latent textures and MLP decoder weights for neural texture compression, matching existing quality.
• RigPAPR: Rig-Based Animation of Static Neural Point Clouds from a Fixed-Viewpoint Video — Interpolation-based neural point rendering inherently avoids joint-boundary artifacts in rigged animation, unlike fixed-shape primitives.
• GrainGS: Gradient-Decoupled Gaussian Splatting for Efficient Dynamic Novel View Synthesis — Decoupling canonical geometry from dynamic deformation and appearance gradients improves 3D Gaussian Splatting stability and quality.
• How AI‍ Learned‌ to‌ Render Photorealistic​ Worlds in Real-Tim‍e And‌ What Th‍at​ Means for the… — Modern photorealistic real-time rendering stems from a paradigm shift to learned models and differentiable image formation.
• Intrinsic decomposition and editing of 3D Gaussian splats — Intrinsic decomposition for 3D Gaussian splats enables texture editing independent of lighting across viewpoints.

See Counsel's argued verdicts on the open AI decisions leaders are weighing →

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.AI updates on arXiv.org.