对比、模仿、适应：从原始人类视频中学习机器人技能
Contrast, Imitate, Adapt: Learning Robotic Skills From Raw Human Videos
AI总结：
本文提出三阶段框架 CIA，通过视频对比学习任务先验、轨迹模仿学习动作先验，并借助有限交互适应新场景，在真实任务上显著提升成功率与泛化性。
摘要：
从原始人类视频中学习机器人技能仍然是一项艰巨挑战。以往工作通过行为克隆或从视频中学习奖励函数来解决该问题。尽管性能出色，但它们可能带来若干问题，例如需要机器人动作、要求人类视频与机器人视频具有一致视角和相似布局，以及样本效率较低。为此，本文的核心洞见是：通过对比视频学习任务先验，通过模仿视频中的轨迹学习动作先验，并利用任务先验引导轨迹适应新场景。我们提出了一个名为 Contrast-Imitate-Adapt（CIA）的三阶段技能学习框架。首先，提出一种交互感知对齐 transformer，通过在时间上对齐视频对来学习任务先验。随后，使用轨迹生成模型学习动作先验。为适应不同于人类视频的新场景，设计了 Inversion-Interaction 方法，用于初始化粗略轨迹并通过有限交互对其进行精化。此外，CIA 引入了一种基于轨迹语义方向的优化方法，以保障交互安全并提高样本效率。由 IAAformer 计算得到的对齐距离被用作奖励。我们在六项真实世界日常任务中评估了 CIA，并通过实验证明，在任务成功率以及对多种新场景布局和物体实例的泛化能力方面，CIA 显著优于以往最先进工作。
英文摘要：
Learning robotic skills from raw human videos remains a non-trivial challenge. Previous works tackled this problem by leveraging behavior cloning or learning reward functions from videos. Despite their remarkable performances, they may introduce several issues, such as the necessity for robot actions, requirements for consistent viewpoints and similar layouts between human and robot videos, as well as low sample efficiency. To this end, our key insight is to learn task priors by contrasting videos and to learn action priors through imitating trajectories from videos, and to utilize the task priors to guide trajectories to adapt to novel scenarios. We propose a three-stage skill learning framework denoted as Contrast-Imitate-Adapt (CIA). An interaction-aware alignment transformer is proposed to learn task priors by temporally aligning video pairs. Then a trajectory generation model is used to learn action priors. To adapt to novel scenarios different from human videos, the Inversion-Interaction method is designed to initialize coarse trajectories and refine them by limited interaction. In addition, CIA introduces an optimization method based on semantic directions of trajectories for interaction security and sample efficiency. The alignment distances computed by IAAformer are used as the rewards. We evaluate CIA in six real-world everyday tasks, and empirically demonstrate that CIA significantly outperforms previous state-of-the-art works in terms of task success rate and generalization to diverse novel scenarios layouts and object instances.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
