PartCrafter：通过组合式潜在扩散Transformer实现结构化3D网格生成
PartCrafter: Structured 3D Mesh Generation via Compositional Latent Diffusion Transformers
AI总结：
PartCrafter提出首个从单张RGB图像端到端生成多部件结构化3D网格的组合式潜在扩散Transformer模型，通过解耦潜在token和层次化注意力实现部件级生成，无需预分割输入，并在可分解3D网格生成任务上优于现有方法。
摘要：
我们提出了PartCrafter，这是第一个结构化3D生成模型，能够从单张RGB图像联合合成多个语义上有意义且几何上不同的3D网格。与现有方法要么生成整体式3D形状、要么采用两阶段流程（即先分割图像再重建每个分割部分）不同，PartCrafter采用统一的组合式生成架构，不依赖预先分割的输入。以单张图像为条件，它同时对多个3D部件进行去噪，从而实现对单个物体和复杂多物体场景的端到端部件感知生成。PartCrafter构建在预训练的3D网格扩散Transformer（DiT）之上，该DiT在完整物体上训练，PartCrafter继承了其预训练权重、编码器和解码器，并引入两项关键创新：（1）组合式潜在空间，其中每个3D部件由一组解耦的潜在token表示；（2）层次化注意力机制，既能在单个部件内部也能跨所有部件实现结构化信息流动，在生成过程中确保全局一致性，同时保留部件级细节。为支持部件级监督，我们通过从大规模3D物体数据集中挖掘部件级标注来构建一个新数据集。实验表明，PartCrafter在生成可分解3D网格方面优于现有方法，包括生成输入图像中不可直接见到的部件，展示了部件感知生成先验在3D理解与合成方面的优势。代码和训练数据将予以发布。
英文摘要：
We introduce PartCrafter, the first structured 3D generative model that jointly synthesizes multiple semantically meaningful and geometrically distinct 3D meshes from a single RGB image. Unlike existing methods that either produce monolithic 3D shapes or follow two-stage pipelines, i.e., first segmenting an image and then reconstructing each segment, PartCrafter adopts a unified, compositional generation architecture that does not rely on pre-segmented inputs. Conditioned on a single image, it simultaneously denoises multiple 3D parts, enabling end-to-end part-aware generation of both individual objects and complex multi-object scenes. PartCrafter builds upon a pretrained 3D mesh diffusion transformer (DiT) trained on whole objects, inheriting the pretrained weights, encoder, and decoder, and introduces two key innovations: (1) A compositional latent space, where each 3D part is represented by a set of disentangled latent tokens; (2) A hierarchical attention mechanism that enables structured information flow both within individual parts and across all parts, ensuring global coherence while preserving part-level detail during generation. To support part-level supervision, we curate a new dataset by mining part-level annotations from large-scale 3D object datasets. Experiments show that PartCrafter outperforms existing approaches in generating decomposable 3D meshes, including parts that are not directly visible in input images, demonstrating the strength of part-aware generative priors for 3D understanding and synthesis. Code and training data will be released.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
