arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~

机器人策略的三手动视觉运动模仿学习

Tri-Manual Visuomotor Imitation Learning of Robot Policies

James Zhao, Mingyuan Ba, Weiming Zhi

arXiv 2607.25731首次发表:更新:

AI 中文总结

研究针对三手动系统中操作员与机器人控制通道数量不匹配问题,提出TriManPolicy系统,核心是依赖感知三臂调度(DATS),能离线重调演示时间,训练单一同步策略,在现实任务中使策略协调更高效,改变手臂监督方式。

AI 中文摘要

双手遥操作是收集机器人演示的有效方式,但它假设操作员和机器人的同步控制通道数量匹配。对于三手动系统,这一假设不成立,因为机器人可同时协调三个手臂,而单个操作员只能连续控制两个。成对模式切换可能会顺序记录原本独立的动作,导致行为克隆再现接口带来的延迟而非任务所需的延迟。我们提出了TriManPolicy,一种三手动模仿学习系统,允许一个操作员为三个手臂演示行为。其核心组件是依赖感知三臂调度(DATS)。关键思想是在重新考虑演示动作发生时间的同时保留它们。DATS通过保留固定持续时间的局部感觉运动片段并根据人类审查的任务顺序和手臂使用约束重新定位它们,在离线时重新调整演示时间。由此产生的数据训练出一个适用于所有三个手臂的单一同步策略,而部署时既不需要依赖图也不需要调度器。在现实世界中执行的六个具有挑战性的任务中,用DATS重新调整时间的演示训练的策略在保持可比的观察到的任务成功率的同时,表现出更高效的协调。离线分析进一步表明,DATS改变了各手臂之间的监督,而不仅仅是消除空闲期。

英文摘要

Bimanual teleoperation provides an effective way to collect robot demonstrations, but it assumes that the operator and robot have matching numbers of simultaneous control channels. This assumption breaks for tri-manual systems: the robot can coordinate three arms concurrently, whereas a single operator can continuously control only two. Pairwise mode switching may therefore record otherwise independent motions sequentially, causing behaviour cloning to reproduce delays imposed by the interface rather than required by the task. We present TriManPolicy, a tri-manual imitation learning system that allows one operator to demonstrate behaviours for three arms. Its central component is Dependency-Aware Tri-Arm Scheduling (DATS). The key idea is to preserve the demonstrated arm motions while reconsidering when they occur. DATS retimes demonstrations offline by preserving local sensorimotor segments of fixed duration and repositioning them according to constraints on task order and arm usage that are reviewed by a human. The resulting data train a single synchronous policy for all three arms, while deployment requires neither the dependency graph nor the scheduler. Across six challenging tasks performed in the real world, policies trained on demonstrations retimed by DATS exhibit more efficient coordination while maintaining comparable observed task success. Offline analysis further shows that DATS changes the supervision across arms rather than merely removing idle periods. Project videos and additional material are available at https://aus.bot/trimanpolicy/.

Comments9 pages, 9 figures. Project page: https://aus.bot/research/trimanpolicy/ . Equal contribution by James Zhao and Mingyuan Ba. Added the project website and clarified the qualitative figure captions

论文原文

arXiv 摘要页 · PDF 原文 · HTML 原文

↑