物理人机交互过程中从示范、纠正和偏好中统一学习
Unified Learning from Demonstrations, Corrections, and Preferences during Physical Human-Robot Interaction
AI总结：
本文提出一种统一学习示范、纠正和偏好的算法，通过从头学习奖励模型并应用约束优化生成轨迹，使机器人能更准确地从物理交互中学习操作任务。
摘要：
人类可以利用物理交互来教导机械臂。这种物理交互根据任务、用户以及机器人目前所学内容的不同而呈现多种形式。最先进的方法侧重于从单一模态学习，或者通过假设机器人拥有关于人类预期任务的先验信息来结合多种交互类型。相比之下，本文引入了一种算法形式化方法，将示范、纠正和偏好的学习统一起来。我们的方法不对人类想要教给机器人的任务做任何假设；相反，我们通过将人类的输入与附近的备选方案进行比较，从头学习奖励模型。我们首先推导出一个损失函数，训练一组奖励模型以匹配人类的示范、纠正和偏好。反馈的类型和顺序由人类教师决定：我们使机器人能够被动或主动地收集这种反馈。然后，我们应用约束优化将学习到的奖励转化为期望的机器人轨迹。通过仿真和一项用户研究，我们证明了我们提出的方法比现有基线更准确地从物理人类交互中学习操作任务，特别是当机器人面临新的或意外的目标时。用户研究的视频可在：https://youtu.be/FSUJsTYvEKU
英文摘要：
Humans can leverage physical interaction to teach robot arms. This physical interaction takes multiple forms depending on the task, the user, and what the robot has learned so far. State-of-the-art approaches focus on learning from a single modality, or combine multiple interaction types by assuming that the robot has prior information about the human's intended task. By contrast, in this paper we introduce an algorithmic formalism that unites learning from demonstrations, corrections, and preferences. Our approach makes no assumptions about the tasks the human wants to teach the robot; instead, we learn a reward model from scratch by comparing the human's inputs to nearby alternatives. We first derive a loss function that trains an ensemble of reward models to match the human's demonstrations, corrections, and preferences. The type and order of feedback is up to the human teacher: we enable the robot to collect this feedback passively or actively. We then apply constrained optimization to convert our learned reward into a desired robot trajectory. Through simulations and a user study we demonstrate that our proposed approach more accurately learns manipulation tasks from physical human interaction than existing baselines, particularly when the robot is faced with new or unexpected objectives. Videos of our user study are available at: https://youtu.be/FSUJsTYvEKU
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
