arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2607.04652cs.RO

KAM-WM:基于潜在世界模型的机器人操作运动学可供性地图

KAM-WM: Kinematic Affordance Maps from Latent World Models for Robot Manipulation

Tsinghua Shenzhen International Graduate School(清华大学深圳国际研究生院);Huawei Technologies Ltd.(华为技术有限公司)

Xinyu Shao, Keru Zhou, Guowei Huang, Yajun Gao, Tongtong Cao, Xiu Li

AI总结:

研究如何从少量演示中学习操作,提出KAM-WM框架,从冻结的潜在视频世界模型提取粗粒度方向交互线索,转化为运动学可供性地图,结合其他信息控制机器人,在实验中取得较好成果。

摘要:

从少量演示中学习操作需要视觉先验,不仅能捕捉交互位置,还能确定交互方式,静态先验无法做到。我们提出KAM-WM框架,从冻结的潜在视频世界模型中提取粗粒度方向交互线索,将其单步潜在速度解释为运动学可供性地图,为任务提供交互区域和粗运动结构,在实验中取得较好效果。

英文摘要:

Learning manipulation from few demonstrations requires visual priors that capture not only where to interact, but also how the interaction should begin; static priors such as segmentation masks encode only the former. We present KAM-WM, a framework that extracts a coarse directional interaction cue from a frozen latent video world model without rollout or world-model fine-tuning. KAM-WM queries a Flow Matching image-to-video backbone once and interprets its single-step latent velocity as a Kinematic Affordance Map (KAM), which provides task-conditioned interaction regions and coarse motion structure. A lightweight Perceiver compresses KAM into tokens that condition a diffusion policy together with RGB observations and proprioception. Across LIBERO and RoboTwin2.0, KAM-WM reaches 90.6% average success on LIBERO and achieves 65.7% and 22.4% success rates in the Easy and Hard settings on RoboTwin2.0, respectively. Controlled comparisons against a zero-order mask prior suggest that part of the gains comes from directional information beyond spatial localization alone. These results indicate that, in the evaluated settings, a frozen video model can provide a useful first-order visual prior for control without the test-time cost of future rollout.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑