Point2Part：基于点提示的统一三维分割
Point2Part: Unified 3D Partitioning from Point Prompts
浏览论文内容
中文总结 AI 辅助
Point2Part提出一种基于点提示的统一三维分割方法，通过联合分割整个形状生成非重叠部件，在图像到部件、网格到部件和部件分割任务上均优于现有方法。
中文摘要 AI 辅助
现有的三维部件分解方法不一定将原始形状分割成非重叠且共同覆盖整个形状的部件，允许重叠或间隙存在，这阻碍了下游的部件级应用。相反，我们将部件分解表述为对整个形状的联合分割，其中预测的部件是非重叠的，并共同恢复整个形状。我们的关键见解是，部件分解应联合考虑所有期望的部件，而不是独立地对每个部件建模。为此，我们开发了一个可提示的模型，用于从图像或网格进行三维部件分解。用户可以通过三维点提示指定期望的部件，以实现可控的分解。给定每个期望部件一个点提示，我们的模型生成相应的部件，作为整个形状的完整分割。我们基于预训练的三维生成模型，首先从输入图像或网格获取形状潜在表示。然后，我们引入一个提示编码器，将每个三维点提示映射为一个部件标记，同时关注形状潜在表示。为了解码期望的部件，我们提出了一种新颖的部件解码器，以从粗到细的方式联合对整个形状与所有部件标记进行评分，将形状体积内的每个位置分配给恰好一个部件。我们在共享的形状潜在空间中执行部件分解，从而实现了图像到部件生成、网格到部件生成和部件分割的统一模型。我们的方法在所有三个任务的所有部件质量指标上均优于现有工作，并将部件之间的兼容性比之前的SOTA方法提高了一个数量级。代码和模型将发布。
英文摘要
Existing 3D part decomposition methods do not necessarily partition the original shape into non-overlapping parts that collectively cover the entire shape, allowing overlaps or gaps that hinder downstream part-level applications. We instead formulate part decomposition as a joint partitioning of the entire shape, where the predicted parts are non-overlapping and jointly recover the entire shape. Our key insight is that part decomposition should consider all desired parts jointly, rather than modeling each part independently. To this end, we develop a promptable model for 3D part decomposition from images or meshes. Users can specify desired parts through 3D point prompts for controllable decomposition. Given one point prompt per desired part, our model produces the corresponding parts as a complete partition of the entire shape. We build on a pretrained 3D generation model and first obtain a shape latent from either an input image or mesh. We then introduce a prompt encoder that maps each 3D point prompt to a part token while attending to the shape latent. To decode the desired parts, we propose a novel part decoder jointly scoring the entire shape against all part tokens in a coarse-to-fine manner, assigning every position within the shape volume to exactly one part. We perform part decomposition in this shared shape latent space, enabling a unified model for image-to-part generation, mesh-to-part generation, and part segmentation. Our method outperforms existing works on all part-quality metrics across all three tasks, and improves compatibility among parts by an order of magnitude over previous SOTA methods. Code and models will be released.
发表机构
- Carnegie Mellon University（卡内基梅隆大学）
机构由 AI 辅助整理，请以论文原文为准。
