发表机构
University of Bonn; Lamarr Institute（波恩大学; 拉马尔研究所）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
Contact Flow 是一种与 embodiment 无关的动作表示，可跨人类演示与不同机器人 embodiment 迁移，用于构建能预测合理操纵结果的世界模型，集成到 pipeline 后在相关任务上表现良好。
AI 中文摘要
世界模型为机器人规划提供了一条有前景的路径，它能让智能体在执行动作前想象并验证动作的后果。然而，当前基于视频的世界模型往往难以捕捉支配操纵的物理约束，尤其是接触约束。此外，它们的动作条件通常受限于特定的 embodiment，比如平行夹爪。我们提出 Contact Flow，这是一种与 embodiment 无关的动作表示，它通过智能体与目标物体之间的 3D 接触点轨迹来编码操纵过程。通过舍弃智能体特定的外观和运动学信息，Contact Flow 为人类演示和机器人执行提供了共享的条件信号。因此，我们可以在 Contact Flow 的条件下，利用人类和机器人的物体交互视频训练一个大规模视频生成模型，从而得到一个能预测物理上合理的操纵结果的世界模型。我们将该模型集成到“提议-想象-验证-执行”的 pipeline 中，在执行前，生成的 rollouts 会由一个视觉语言模型进行评估。在 DROID 数据集和真实世界桌面操纵任务上的实验表明，Contact Flow 能够实现人类演示与不同机器人 embodiment 之间的迁移。
英文摘要
World models offer a promising route toward robot planning by enabling agents to imagine and verify the consequences of actions before execution. However, current video-based world models often struggle to capture the physical constraints that govern manipulation, particularly contact. Further, their action conditioning is often constrained to specific embodiments such as parallel grippers. We propose \emph{Contact Flow}, an embodiment-agnostic action representation that encodes manipulation through the trajectory of 3D contact points between an actor and a target object. By discarding actor-specific appearance and kinematics, Contact Flow provides a shared conditioning signal for both human demonstrations and robotic execution. Therefore, we can train a large-scale video generative model on both human and robotic object interaction videos conditioned on Contact Flow, yielding a world model that predicts physically plausible manipulation outcomes. We integrate this model into a propose-imagine-verify-act pipeline, where generated rollouts are assessed by a vision-language model before execution. Experiments on the DROID dataset and real-world tabletop manipulation tasks demonstrate that Contact Flow enables transfer between human demonstrations and different robotic embodiments.
