arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2605.30347cs.CVcs.GR
收藏

NeuROK:生成式4D神经物体运动学

NeuROK: Generative 4D Neural Object Kinematics

Stanford University(斯坦福大学);University of Cambridge(剑桥大学);Cornell University(康奈尔大学)

Chen Geng, Guangzhao He, Yue Gao, Yunzhi Zhang, Shangzhe Wu, Jiajun Wu

AI总结:

提出基于Transformer的编码器-解码器模型NeuROK,通过学习物体潜在运动学参数化,实现从静态3D物体生成逼真的4D动态变形,克服了传统方法对预定义物理模型和特定类别的依赖。

摘要:

数据驱动的方法已经彻底改变了3D视觉,使Transformer能够有效地重建和生成静态3D物体。然而,生成模拟4D动态——即静态物体在各种物理条件下的逼真时间变形——仍然具有挑战性且通常是特设的,尽管它在构建全面的3D世界模型中很重要。大多数现有方法假设一个预定义的物理模型并使用系统辨识来估计参数,将这些方法限制在特定类别和小规模数据集上。我们提出,通过学习物体中心物理系统的数据驱动运动学状态参数化,可以克服这些限制。具体来说,我们学习了一个表示物体所有可能状态的潜在空间,以及一个将任何采样的潜在向量映射到物体合理变形形状的解码器。我们将这种参数化称为神经物体运动学(NeuROK),并在精心策划的大规模4D数据集上训练基于Transformer的编码器-解码器模型。这种公式化和学习到的模型显著简化了模拟动态的生成,因为我们只需要从经典物理中拉格朗日力学的角度考虑低维潜在空间内的动态。我们在各种动态物体类型上展示了这种神经模拟框架的有效性和通用性,显示出相对于先前工作的明显优势。项目页面:https://chen-geng.com/neurok

英文摘要:

Data-driven approaches have revolutionized 3D vision, enabling transformers to effectively reconstruct and generate static 3D objects. However, generating simulative 4D dynamics -- realistic temporal deformations of static objects under various physical conditions -- remains challenging and often ad hoc, despite its importance in building comprehensive 3D world models. Most existing methods assume a predefined physical model and use system identification to estimate parameters, restricting these methods to specific categories and small-scale datasets. We propose that these restrictions can be overcome by learning a data-driven kinematic state parameterization for object-centric physical systems. Specifically, we learn both a latent space representing all possible states of the object and a decoder that maps any sampled latent to a plausibly deformed shape of the object. We refer to this parameterization as Neural Object Kinematics (NeuROK), and learn a transformer-based encoder-decoder model on a curated large-scale 4D dataset. This formulation and the learned model significantly simplify the generation of simulative dynamics since we only need to consider the dynamics within a low-dimensional latent space from the Lagrangian mechanics' perspective in classical physics. We demonstrate the effectiveness and generality of this neural simulation framework across diverse dynamic object types, showing clear advantages over prior works. Project page: https://chen-geng.com/neurok

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑