arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2601.18323cs.RO
收藏

TC-IDM:为可执行零样本机器人运动实现视频生成

TC-IDM: Grounding Video Generation for Executable Zero-shot Robot Motion

Beijing Innovation Center of Humanoid Robotics(北京人形机器人创新中心);State Key Laboratory of Multimedia Information Processing(多媒体信息处理国家重点实验室);School of Computer Science, Peking University(北京大学计算机科学学院)

Weishi Mi, Yong Bao, Xiaowei Chi, Xiaozhu Ju, Zhiyuan Qin, Kuangzhi Ge, Kai Tang, Peidong Jia, Shanghang Zhang, Jian Tang

AI总结:

TC-IDM通过工具中心逆动力学模型实现视频生成,提升机器人零样本任务的执行能力与泛化性能。

摘要:

视觉-语言-动作(VLA)范式通过利用视觉语言模型实现了强大的机器人控制,但其依赖大规模高质量机器人数据限制了泛化能力。生成式世界模型为通用具身AI提供了有前途的替代方案,但其像素级计划与物理可执行动作之间仍存在关键差距。为此,我们提出了工具中心逆动力学模型(TC-IDM)。通过聚焦由世界模型合成的工具想象轨迹,TC-IDM建立了一个稳健的中间表示,弥合了视觉规划与物理控制之间的差距。TC-IDM通过分割和3D运动估计从生成的视频中提取工具的点云轨迹。考虑到多样化的工具属性,我们的架构采用解耦的动作头将这些计划轨迹投影到6自由度末端执行器运动和相应的控制信号。这种计划和翻译范式不仅支持广泛的末端执行器,还显著提高了视角不变性。此外,它在长时间跨度和分布外任务中表现出强大的泛化能力,包括与变形物体的交互。在现实世界评估中,具有TC-IDM的世界模型实现了61.11%的平均成功率,其中简单任务为77.7%,零样本变形物体任务为38.46%。它明显优于端到端VLA风格基线和其他逆动力学模型。

英文摘要:

The vision-language-action (VLA) paradigm has enabled powerful robotic control by leveraging vision-language models, but its reliance on large-scale, high-quality robot data limits its generalization. Generative world models offer a promising alternative for general-purpose embodied AI, yet a critical gap remains between their pixel-level plans and physically executable actions. To this end, we propose the Tool-Centric Inverse Dynamics Model (TC-IDM). By focusing on the tool's imagined trajectory as synthesized by the world model, TC-IDM establishes a robust intermediate representation that bridges the gap between visual planning and physical control. TC-IDM extracts the tool's point cloud trajectories via segmentation and 3D motion estimation from generated videos. Considering diverse tool attributes, our architecture employs decoupled action heads to project these planned trajectories into 6-DoF end-effector motions and corresponding control signals. This plan-and-translate paradigm not only supports a wide range of end-effectors but also significantly improves viewpoint invariance. Furthermore, it exhibits strong generalization capabilities across long-horizon and out-of-distribution tasks, including interacting with deformable objects. In real-world evaluations, the world model with TC-IDM achieves an average success rate of 61.11 percent, with 77.7 percent on simple tasks and 38.46 percent on zero-shot deformable object tasks. It substantially outperforms end-to-end VLA-style baselines and other inverse dynamics models.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑