PartMat：基于单个全局隐变量的材质感知三维部件分解

中文摘要 AI 辅助

部件级三维生成近年受关注，可生成结构化、可编辑的三维资产，但现有方法通常按功能语义分解物体，而非室内设计等实际三维应用所需的可编辑材质边界（如织物、木材、金属），且当前方法常独立生成部件，导致计算成本随部件数量线性缩放。为解决这些局限，本文提出PartMat，一种高效的材质感知三维部件分解流水线，用单个全局隐变量表示多部件几何。给定参考图像和单个完整物体几何，PartMat将物体分解为符合材质边界的部件：首先，提出PartVAE学习该统一表示并在单次前向传播中解码所有材质部件，从而将推理成本与部件数量解耦；其次，基于该表示训练扩散模型生成部件，并通过强化学习优化以实现准确的材质分配与重叠抑制；最后，为恢复细粒度几何细节，引入带部件注意力的稀疏体素流匹配模型进行几何后处理。大量实验表明，PartMat在材质感知分解准确率上显著优于现有基线，同时实现了相当的几何质量并保持高效推理。

英文摘要

Part-level 3D generation has recently attracted increasing attention for producing structured and editable 3D assets. However, existing methods typically decompose objects according to functional semantics rather than the editable material boundaries (e.g., fabric, wood, metal) required in practical 3D applications such as interior design. Additionally, current methods often generate parts independently, causing computational costs to scale linearly with the part count. To address these limitations, we present PartMat, an efficient material-aware 3D part decomposition pipeline that represents multi-part geometry with a single global latent. Given a reference image and a single whole-object geometry, PartMat decomposes the object into parts that follow material boundaries. First, we propose PartVAE to learn such a unified representation and decode all material parts in a single forward pass, thereby decoupling inference cost from the number of parts. Second, with this representation, a diffusion model is trained for part generation and refined via reinforcement learning for accurate material assignment and overlap suppression. Finally, to recover fine-grained geometric details, we introduce a sparse-voxel flow-matching model with part attention for geometry post-processing. Extensive experiments demonstrate that PartMat significantly outperforms existing baselines in material-aware decomposition accuracy and achieves comparable geometric quality, while maintaining efficient inference.

发表机构

• Alibaba Group（阿里巴巴集团）

机构由 AI 辅助整理，请以论文原文为准。