You Only Teach Once：从视频演示中学习单样本双臂机器人操作
You Only Teach Once: Learn One-Shot Bimanual Robotic Manipulation from Video Demonstrations
AI总结：
针对双臂机器人操作挑战，提出YOTO框架，从单次双目视频演示中提取动作模式并生成多样化训练数据，学习定制化双臂扩散策略，在5个复杂任务中实现高精度与强泛化。
摘要：
双臂机器人操作由于具有双臂时空协调和高维动作空间的特征，一直是具身智能领域的长期挑战。以往的研究依赖预定义的动作分类法或直接遥操作来缓解或规避这些问题，这往往使其缺乏简洁性、通用性和可扩展性。不同地，我们认为教授双臂操作最有效且高效的方法是从人类演示视频中学习，其中丰富的特征（如时空位置、动态姿态、交互状态和灵巧转换）几乎可以免费获取。在本工作中，我们提出了YOTO（You Only Teach Once），它能从少至单次手部运动的双目观测中提取并注入双臂动作模式，从而教授双机器人手臂各种复杂任务。此外，基于关键帧的运动轨迹，我们设计了一种精细的解决方案，用于快速生成具有被操作物体及其位置多样变化的训练演示。这些数据随后可用于学习跨多种场景的定制化双臂扩散策略（BiDP）。在实验中，YOTO在模仿5个复杂的长期双臂任务中取得了令人瞩目的表现，在不同视觉和空间条件下具备强大的泛化能力，并在准确性和效率上优于现有的视觉运动模仿学习方法。我们的项目链接是 https://hnuzhy.github.io/projects/YOTO。
英文摘要：
Bimanual robotic manipulation is a long-standing challenge of embodied intelligence due to its characteristics of dual-arm spatial-temporal coordination and high-dimensional action spaces. Previous studies rely on pre-defined action taxonomies or direct teleoperation to alleviate or circumvent these issues, often making them lack simplicity, versatility and scalability. Differently, we believe that the most effective and efficient way for teaching bimanual manipulation is learning from human demonstrated videos, where rich features such as spatial-temporal positions, dynamic postures, interaction states and dexterous transitions are available almost for free. In this work, we propose the YOTO (You Only Teach Once), which can extract and then inject patterns of bimanual actions from as few as a single binocular observation of hand movements, and teach dual robot arms various complex tasks. Furthermore, based on keyframes-based motion trajectories, we devise a subtle solution for rapidly generating training demonstrations with diverse variations of manipulated objects and their locations. These data can then be used to learn a customized bimanual diffusion policy (BiDP) across diverse scenes. In experiments, YOTO achieves impressive performance in mimicking 5 intricate long-horizon bimanual tasks, possesses strong generalization under different visual and spatial conditions, and outperforms existing visuomotor imitation learning methods in accuracy and efficiency. Our project link is https://hnuzhy.github.io/projects/YOTO.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
