arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2610.23002cs.RO
收藏

机器人时空编辑:通过掩码动作重建实现可控动作生成的统一接口

Robotic Spatiotemporal Editing: A Unified Interface for Controllable Action Generation via Masked Action Reconstruction

Tsinghua University(清华大学);CocoMatrix;Shanghai Qizhi Institute(上海期智研究院);UIUC(伊利诺伊大学厄巴纳-香槟分校)

Yanbang Li, Bu He, Yi Tian, Yu-Chiao Huang, Chenyu Zhang, Ruoqu Chen, Yuanzhe Liu, Yuxiang Gao, Mengdi Xu

AI总结:

该研究提出MAP接口与Inpainting Policy,通过掩码动作重建统一时空控制参考,在ManiSkill等数据集及现实实验中实现高成功率与低误差的机器人可控动作生成。

摘要:

生成策略可合成机器人行为,但通过多样时空参考对其进行控制与编辑仍具挑战性。现实中,人类意图要求机器人行为满足动作执行的位置、时间及方式约束。我们的核心见解是,时空控制参考(如关键点、光流、方向和关键位姿)可视为同一时间笛卡尔动作空间的部分规范。我们引入MAP(Masked Action Space,掩码动作空间)接口以统一这些动作参考,并将可控操作表述为动作修复问题。我们的修复策略(Inpainting Policy,IP)重建末端执行器缺失的动作维度,结合进度引导对齐与多掩码训练,以支持未见过的动作-约束组合。我们进一步采用无训练的RePaint变体,在去噪过程中迭代修复动作以降低控制误差。在ManiSkill和LIBERO-10数据集上,我们的策略分别达到92.67%和64.67%的平均任务成功率。现实世界实验表明,其时空控制的平均位置误差为44.3 mm,旋转误差为7°,证明该策略可利用多样参考引导可执行的机器人行为。

英文摘要:

Generative policies can synthesize robot behaviors, yet controlling and editing them through diverse spatiotemporal references remains challenging. In the real world, human intent requires robot behaviors that satisfy constraints on where, when, and how actions are executed. Our key insight is that spatiotemporal control references, such as keypoints, flows, orientations, and keyposes, can be viewed as partial specifications of the same temporal Cartesian action space. We introduce the MAP (Masked Action Space) interface to unify these action references and formulate controllable manipulation as an action inpainting problem. Our Inpainting Policy (IP) reconstructs missing action dimensions of the end-effector, combining progress-guided alignment with multi-mask training supporting unseen action-constraint combinations. We further adopt a training-free RePaint variant that iteratively inpaints the action during denoising to reduce control error. On ManiSkill and LIBERO-10, our policy achieves average task success rates of 92.67% and 64.67%, respectively.Real-world experiments demonstrate spatiotemporal control with average positional and rotational errors of 44.3 mm and 7°, respectively, showing that our policy can use diverse references to guide executable robot behaviors.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑