UniSHARP: Universal Sharp Monocular View Synthesis

2026-06-08 · Source: cs.CV updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems · Depth: Expert, extended

Summary

UniSHARP extends the popular SHARP photorealistic view synthesis method for universal monocular rendering across a continuum of camera systems, including conventional perspective, wide-field-of-view, fisheye, and omnidirectional panoramic settings. It overcomes SHARP's pinhole-specific assumptions by aligning various images in a unified omnidirectional latent space, performing implicit alignment in both feature and Gaussian spaces. Gaussian primitives are arranged along rays and radial distances in a ray-based universal representation, while 2D semantic and 3D spatial features from UniK3D-inspired encoders generate the complete Gaussian cloud. The method was evaluated on a new FoV-stratified benchmark covering diverse imaging systems from 60° to 360° FoV, demonstrating superior performance over alternative methods and supporting pose-free monocular inference.

Key takeaway

For Machine Learning Engineers developing 3D vision systems for diverse camera inputs, you should consider UniSHARP's ray-distance Gaussian representation to overcome the limitations of pinhole-specific methods like SHARP. This approach allows for robust, high-fidelity novel view synthesis from single images across perspective, wide-FoV, fisheye, and panoramic cameras, even without explicit camera calibration, broadening application possibilities.

Key insights

UniSHARP enables universal monocular novel view synthesis across diverse camera types using a unified ray-distance Gaussian representation.

Principles

• Unified ray-distance space decouples camera projection from scene representation.
• Fusing 2D semantic and 3D geometric features improves high-fidelity synthesis.
• Mixed-camera training with distortion adaptation enhances robustness.

Method

UniSHARP predicts 3D Gaussian primitives by constructing Geometry Anchored Gaussians in ray-distance space, then adds Feature Conditioned Gaussian residuals from fused 2D/3D features.

In practice

• Apply to AR/VR content creation and immersive telepresence.
• Use for robotic navigation and 3D content generation.
• Enable pose-free monocular inference from single RGB images.

Topics

• Monocular View Synthesis
• 3D Gaussian Splatting
• Universal Camera Models
• Omnidirectional Imaging
• Ray-Distance Representation
• AR/VR
• Neural Radiance Fields

Best for: Research Scientist, AI Scientist, Machine Learning Engineer, Computer Vision Engineer

Related on AIssential

• UniSHARP: Universal Sharp Monocular View Synthesis — UniSHARP unifies monocular view synthesis across diverse camera types via implicit alignment in an omnidirectional latent space.
• WildSplatter: Feed-forward 3D Gaussian Splatting with Appearance Control from Unconstrained Images — WildSplatter enables fast, pose-free 3DGS from unconstrained images by disentangling geometry and appearance via learned embeddings.
• UniVL: Unified Vision-Language Embedding for Spatially Grounded Contextual Image Generation — UniVL unifies spatial and semantic image generation conditioning by rendering text into masks, eliminating separate text encoders.
• Honey, I Shrunk the Arc de Triomphe! — A new dataset and method mitigate scale-collapse in monocular depth estimation for distant, unconstrained scenes.
• Utonia: Toward One Encoder for All Point Clouds — Utonia unifies diverse point cloud domains into a single self-supervised transformer encoder for consistent representation learning.
• TROPHIES: Temporal Reconstruction of Places, Humans, and Cameras from Multi-view Videos — TROPHIES unifies human, scene, and camera reconstruction from multi-view videos into a globally consistent 4D space.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CV updates on arXiv.org.