Kinema4D: 用于空间时间具身模拟的运动学4D世界建模
Kinema4D: Kinematic 4D World Modeling for Spatiotemporal Embodied Simulation
AI总结：
本文提出Kinema4D，一种基于动作条件的4D生成机器人模拟器，通过精确的4D机器人控制表示和环境反应的生成建模，实现了高精度的空间时间交互模拟，并首次展示零样本迁移能力。
摘要：
模拟机器人与世界互动是具身AI的核心。最近，一些工作展示了利用视频生成超越传统模拟器刚性视觉/物理约束的潜力。然而，它们主要在2D空间中操作或依赖静态环境线索，忽略了机器人与世界互动本质上是需要精确交互建模的4D时空事件。为恢复这种4D本质并确保精确机器人控制，我们引入了Kinema4D，一种新的动作条件的4D生成机器人模拟器，将机器人与世界互动分解为：i) 精确的4D机器人控制表示：通过运动学驱动基于URDF的3D机器人，产生精确的4D机器人控制轨迹；ii) 环境反应的生成4D建模：将4D机器人轨迹投影为点图作为时空视觉信号，控制生成模型合成复杂环境的反应动力学，生成同步的RGB/点图序列。为促进训练，我们精心编纂了一个大规模数据集，称为Robo4D-200k，包含201,426个机器人交互片段，具有高质量的4D注释。大量实验表明，我们的方法能够有效模拟物理上合理、几何一致且具身无关的交互，真实反映多样化的现实世界动态。首次，它展示了潜在的零样本迁移能力，为下一代具身模拟的发展提供了高保真的基础。
英文摘要：
Simulating robot-world interactions is a cornerstone of Embodied AI. Recently, a few works have shown promise in leveraging video generations to transcend the rigid visual/physical constraints of traditional simulators. However, they primarily operate in 2D space or are guided by static environmental cues, ignoring the fundamental reality that robot-world interactions are inherently 4D spatiotemporal events that require precise interactive modeling. To restore this 4D essence while ensuring the precise robot control, we introduce Kinema4D, a new action-conditioned 4D generative robotic simulator that disentangles the robot-world interaction into: i) Precise 4D representation of robot controls: we drive a URDF-based 3D robot via kinematics, producing a precise 4D robot control trajectory. ii) Generative 4D modeling of environmental reactions: we project the 4D robot trajectory into a pointmap as a spatiotemporal visual signal, controlling the generative model to synthesize complex environments' reactive dynamics into synchronized RGB/pointmap sequences. To facilitate training, we curated a large-scale dataset called Robo4D-200k, comprising 201,426 robot interaction episodes with high-quality 4D annotations. Extensive experiments demonstrate that our method effectively simulates physically-plausible, geometry-consistent, and embodiment-agnostic interactions that faithfully mirror diverse real-world dynamics. For the first time, it shows potential zero-shot transfer capability, providing a high-fidelity foundation for advancing next-generation embodied simulation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
