Data-driven approaches have revolutionized 3D vision, enabling transformers to effectively reconstruct and generate static 3D objects. However, generating simulative 4D dynamics -- realistic temporal deformations of static objects under various physical conditions -- remains challenging and often ad hoc, despite its importance in building comprehensive 3D world models. Most existing methods assume a predefined physical model and use system identification to estimate parameters, restricting these methods to specific categories and small-scale datasets. We propose that these restrictions can be overcome by learning a data-driven kinematic state parameterization for object-centric physical systems. Specifically, we learn both a latent space representing all possible states of the object and a decoder that maps any sampled latent to a plausibly deformed shape of the object. We refer to this parameterization as Neural Object Kinematics (NeuROK), and learn a transformer-based encoder-decoder model on a curated large-scale 4D dataset. This formulation and the learned model significantly simplify the generation of simulative dynamics since we only need to consider the dynamics within a low-dimensional latent space from the Lagrangian mechanics' perspective in classical physics. We demonstrate the effectiveness and generality of this neural simulation framework across diverse dynamic object types, showing clear advantages over prior works. Project page: https://chen-geng.com/neurok
EA-WM: Event-Aware Generative World Model with Structured Kinematic-to-Visual Action Fields
Fudan University(复旦大学);Zhongguancun Academy(中关村学院);Zhongguancun Institute of Artificial Intelligence(中关村人工智能研究院);University of Science and Technology of China(中国科学技术大学);DeepCybo(深瞳)
KiGRAS: Kinematic-Driven Generative Model for Realistic Agent Simulation
University of Science and Technology of China(中国科学技术大学);Mach Drive(马赫动力);Tsinghua Shenzhen International Graduate School, Tsinghua University(清华大学深圳国际研究生院)
TopoCap: Learning Topology-Agnostic Motion Priors for Monocular Video-to-Animation
Zhili College, Tsinghua University(清华大学致理书院);BNRist, Department of Computer Science and Technology, Tsinghua University(清华大学计算机科学与技术系,北京国家信息科学与技术研究中心);VAST
LAD-Drive: Bridging Language and Trajectory with Action-Aware Diffusion Transformers
Institute for Intelligent Systems, Esslingen University of Applied Sciences(智能系统研究所,埃斯林根应用科学大学);Department of Computer Science, University of Freiburg(计算机科学系,弗赖堡大学)
DICArt: Advancing Category-level Articulated Object Pose Estimation in Discrete State-Spaces
University of Science and Technology of China(中国科学技术大学);Shanghai Jiao Tong University(上海交通大学);Zhejiang University(浙江大学);East China Normal University(华东师范大学);Peking University(北京大学);Emory University(埃默里大学);Hefei University of Technology(合肥工业大学);Jianghuai Advance Technology Center(江淮先进技术中心);Anhui Provincial Key Laboratory of Humanoid Robots(安徽省人形机器人重点实验室)