PWM-ArtGen: 面向铰接物体生成的部分世界模型
PWM-ArtGen: Part World Model for Articulated Object Generation
AI总结:
提出部分世界模型PWM-ArtGen,通过联合学习视觉动态和运动学参数,利用无标注数据协同训练,实现从单张图像生成铰接3D物体,在静止状态和零样本泛化上显著优于现有方法。
摘要:
从单张图像生成铰接3D物体的关键挑战在于准确预测潜在的运动学结构。现有方法要么直接从缺乏动态部件级运动学关系的静态图像中推断运动学参数,要么从单张图像生成的视觉动态中估计参数,这容易导致两步累积误差。此外,现有标注数据集的规模和多样性有限,进一步阻碍了对复杂真实世界物体的泛化。为克服这些限制,我们提出学习视觉动态和运动学参数的联合分布。认识到铰接物体可以表述为动态系统,我们提出了一个统一的部分世界模型PWM-ArtGen。为利用无标注数据,该模型将动作扩散和图像扩散与独立的扩散时间步耦合,从而实现视觉分支的协同训练。我们进一步整理了一个包含19.7k个部件级图像对的无运动学标注的光真实感数据集,以支持协同训练。实验表明,PWM-ArtGen在静止状态下显著优于现有基线,并对分布外物体展现出强大的零样本泛化能力。
英文摘要:
The key challenge in articulated 3D object generation from a single image is accurately predicting the underlying kinematic structure. Existing methods either infer kinematic parameters directly from a static image that lacks dynamic part-level kinematic relationships, or estimate parameters from visual dynamics generated from a single image, which is prone to accumulated errors of two steps. Moreover, the limited scale and diversity of existing annotated datasets further hinder generalization to complex, real-world objects. To overcome these limitations, we propose to learn the joint distribution of visual dynamics and kinematic parameters. Recognizing that articulated objects can be formulated as dynamic systems, we propose a unified Part World Model called PWM-ArtGen. To leverage unannotated data, this model couples action diffusion and image diffusion with independent diffusion timesteps, which enables visual branch co-training. We further curate a photorealistic dataset of 19.7k part-level image pairs without kinematic annotations, to support co-training. Experiments demonstrate that PWM-ArtGen substantially outperforms existing baselines in the resting state and exhibits strong zero-shot generalization to out-of-distribution objects.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。