Harnessing Generative Image Models for Training-Free Primitive Shape Abstraction

2026-05-30 · Source: cs.CV updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems · Depth: Expert, extended

Summary

A novel training-free pipeline enables semantic 3D shape abstraction by harnessing large-scale generative image models. This method renders multi-view images of a 3D object, uses a vision-language model to identify semantic parts, and prompts a generative image model to create color-coded segmentation masks. These masks are then reprojected onto the 3D geometry, clustered, and fitted with superquadric primitives via parameter optimization. The approach contains no learned parameters, making it category-agnostic and orientation-invariant. It achieves the lowest Chamfer distance on HumanPrim (0.079) and Toys4K (0.093) datasets, using 5-9 primitives per object on average. A ground-truth segmentation study confirms that part segmentation, not primitive fitting, is the current accuracy bottleneck, indicating future improvements in generative models will directly enhance this pipeline's performance.

Key takeaway

For AI Scientists developing 3D understanding systems, this work demonstrates a powerful paradigm shift: you can achieve robust, category-agnostic shape abstraction without task-specific training. Focus on integrating advanced generative image models for superior semantic segmentation, as your abstraction quality will directly inherit their future improvements. Consider adaptive view selection or ensemble segmentation to further enhance robustness and coverage for complex geometries.

Key insights

Training-free 3D shape abstraction leverages generative image models for semantic part segmentation and classical optimization for primitive fitting.

Principles

• Abstraction quality scales with generative model improvements.
• Semantic segmentation is critical for accurate primitive fitting.
• Coupling foundation models with classical optimization is effective.

Method

The pipeline involves multi-view rendering, VLM analysis for semantic parts, generative image model mask generation, 3D reprojection, color-restricted clustering, and parallel multi-start superquadric fitting via Chamfer distance optimization.

In practice

• Apply to robotics for grasping and collision detection.
• Use for semantic scene understanding.
• Generate compact 3D representations for shape editing.

Topics

• 3D Shape Abstraction
• Generative Image Models
• Vision-Language Models
• Superquadrics
• Training-Free Learning
• Chamfer Distance

Best for: Research Scientist, AI Scientist, Computer Vision Engineer, Robotics Engineer

Related on AIssential

• Harnessing Generative Image Models for Training-Free Primitive Shape Abstraction — Pretrained generative image models can be directly harnessed for training-free 3D primitive shape abstraction, bypassing fine-tuning.
• Texture-Shape Bias Balancing for Robust Synthetic-to-Real Semantic Segmentation in Automotive NIR Imagery — A generative augmentation framework reduces the synthetic-to-real domain gap for automotive NIR semantic segmentation by balancing texture-shape bias.
• Training-Free Fine-Grained Semantic Segmentations in Low Data Regimes: A FungiTastic Baseline — FungiTastic offers a training-free, two-stage framework using SAM3 and DINOv3 for fine-grained semantic segmentation in low-data settings.
• EditSSC: Toward Editable Semantic Occupancy Scenes with Unconditional Diffusion Models — EditSSC repurposes 2D latent diffusion for 3D semantic scene generation, enabling training-free editing capabilities.
• Preserve the Hard, Regenerate the Rest: Uncertainty-Guided Synthetic Training Data Augmentation with Diffusion Models — Improve semantic segmentation by augmenting uncertain regions with diffusion models, preserving original labels.
• Unsupervised Pixel-Level Semantic Left-Right Understanding of In-the-Wild Images — An unsupervised framework predicts pixel-level semantic left-right understanding in diverse images by combining 3D shape and image datasets.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CV updates on arXiv.org.