SuperVoxelGPT: Adaptive and Ordered 3D Tokenization for Autoregressive Shape Generation

2026-07-08 · Source: cs.CV updates on arXiv.org · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Robotics & Autonomous Systems · Depth: Expert, extended

Summary

SuperVoxelGPT is a novel representation-first framework designed for high-resolution 3D shape generation using autoregressive multimodal large language models (MLLMs). It addresses the challenge of scaling MLLMs to complex 3D shapes by introducing adaptive and deterministically ordered supervoxel tokenization. The framework first predicts a coarse geometric saliency distribution from a prompt, then constructs a shape-adaptive supervoxel partition using saliency-guided centroidal Voronoi tessellation (CVT), allocating fine-grained cells to complex regions and larger cells to smooth areas. A SuperVoxelVAE encodes local geometry into discrete tokens, and a fine-tuned MLLM (Qwen2.5-0.5B) autoregressively generates these tokens, leveraging Jacobi decoding for parallel inference. Experiments on Trellis-500K demonstrate that SuperVoxelGPT reduces token sequence length to 12.8% of uniform voxel tokenization, achieves state-of-the-art generation quality, and delivers an average 10x speedup over prior methods for 1024^3 resolution shapes.

Key takeaway

For AI Architects and Machine Learning Engineers developing high-resolution 3D generative models, SuperVoxelGPT offers a significant advancement. You should consider implementing adaptive supervoxel tokenization to drastically reduce sequence length and achieve a 10x inference speedup without sacrificing quality. This approach allows you to scale MLLM-based 3D generation to complex geometries more efficiently, making high-fidelity asset creation practical. Evaluate its saliency prediction dependency for your specific detail requirements.

Key insights

Adaptive, ordered supervoxel tokenization resolves the trade-off between compactness and spatial fidelity for autoregressive 3D generation.

Principles

• Deterministic sequential structure is crucial for stable autoregressive modeling.
• Adaptive capacity allocation improves efficiency and detail preservation.
• Decoupling token allocation from geometry generation enhances scalability.

Method

SuperVoxelGPT uses a two-stage process: first, prompt-to-saliency volume prediction and saliency-guided CVT for adaptive supervoxel partitioning; second, SuperVoxelVAE encoding and MLLM-based Jacobi decoding for token generation.

In practice

• Integrate supervoxel layers into existing sparse voxel VAEs for token compression.
• Apply Jacobi decoding when sequence length is known for parallel inference.

Topics

• 3D Generation
• Multimodal LLMs
• SuperVoxel Tokenization
• Centroidal Voronoi Tessellation
• Jacobi Decoding
• Voxel-based Models

Best for: Research Scientist, Computer Vision Engineer, AI Scientist, Machine Learning Engineer, AI Architect

Related on AIssential

• SuperVoxelGPT: Adaptive and Ordered 3D Tokenization for Autoregressive Shape Generation — SuperVoxelGPT enables efficient, high-resolution 3D shape generation by adaptively tokenizing shapes into ordered supervoxels, resolving prior scaling issues.
• Adaptive Volumetric Mechanical Property Fields Invariant to Resolution — AdaVoMP predicts dense, spatially-varying mechanical properties for 3D objects using an adaptive sparse voxel structure and a transformer model.
• SubdivAR: Autoregressive Next-Scale Prediction for Neural Mesh Subdivision — SubdivAR reformulates mesh subdivision as autoregressive next-scale prediction using a novel Mesh Autoregressive Representation.
• Global-Local Monte Carlo Tree Search in Vision-Language Models for Text-to-3D Indoor Scene Generation — PRM-guided MCTS with hierarchical scene representation enables LVLMs to generate realistic 3D indoor scenes by correcting errors and balancing search.
• Balancing Image Compression and Generation with Bootstrapped Tokenization — SelfBootTok decomposes image tokens into global and local groups, predicting local details from global tokens for efficient generation and scalable training.
• HYDRA-X: Native Unified Multimodal Models with Holistic Visual Tokenizers — Unifying image and video tokenization in a single ViT with specific architectural choices significantly enhances multimodal model performance and editing consistency.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by cs.CV updates on arXiv.org.