Representing 3D shapes as compact sets of geometric primitives is fundamental to robotics, simulation, and scene understanding. Recently, generative image models trained at scale have emerged as generalist visual learners capable of identifying and segmenting object parts directly in the image domain across arbitrary categories without task-specific training.
We explore whether these pretrained models can be harnessed directly, without fine-tuning, and affirmatively answer this question with a training-free harness. Our pipeline renders multi-view images of a 3D object, uses a vision-language model to analyze its semantic parts, prompts a generative image model to paint a color-coded part segmentation mask, reprojects it onto the geometry, and fits a superquadric primitive to each part via parameter optimization.
This approach contains no learned parameters: it is category-agnostic and orientation-invariant, properties that previous learning-based models struggled with. Its accuracy ceiling rises with future generative model improvements, confirmed by a ground-truth segmentation study showing that part segmentation, not primitive fitting, is the current accuracy bottleneck. On HumanPrim and Toys4K, our method achieves the lowest Chamfer distance among all evaluated methods, averaging 5 to 9 primitives per object.
Blogger's Review: This research highlights the potential of generative image models in representing 3D shapes, particularly in a training-free context. By combining vision-language models with generative models, the authors successfully achieve efficient shape abstraction, providing new insights for future robotics and simulation applications. The parameter-free design adds to the method's adaptability across various scenarios, making it noteworthy.