从物理校正序列中学习人类目标
Learning Human Objectives from Sequences of Physical Corrections
摘要:
当个人、辅助和交互式机器人犯错误时,人类会通过物理交互自然而直观地纠正这些错误。在简单的情况下,一个修正就足以传达人类想要的东西。但是,当人类与多个机器人一起工作时,或者机器人正在执行复杂的任务时,人类通常必须进行几次纠正以修复机器人的行为。先前的研究假设这些物理校正中的每一个都是独立的事件,并且每次从它们中学习一个。然而,这错过了关键信息:这些相互作用中的每一个都是相互关联的,只有在一起看才有意义。可替代地,其他工作原因在由所有人类的校正产生的最终轨迹上。但这种方法必须等到任务结束才能从修正中学习,而不是以在线方式从修正中推断。在本文中,我们正式提出了一种方法,在当前任务的物理校正序列的学习。为了做到这一点,我们引入了一个辅助奖励,捕捉人类之间的权衡作出更正,提高机器人的即时奖励和长期性能。我们在远程和面对面的人机实验中评估了所得到的算法,并与独立基线和最终基线进行了比较。我们的研究结果表明,用户最好能够传达他们的目标时,机器人的原因在他们的序列的更正。
英文摘要:
When personal, assistive, and interactive robots make mistakes, humans naturally and intuitively correct those mistakes through physical interaction. In simple situations, one correction is sufficient to convey what the human wants. But when humans are working with multiple robots or the robot is performing an intricate task often the human must make several corrections to fix the robot's behavior. Prior research assumes each of these physical corrections are independent events, and learns from them one-at-a-time. However, this misses out on crucial information: each of these interactions are interconnected, and may only make sense if viewed together. Alternatively, other work reasons over the final trajectory produced by all of the human's corrections. But this method must wait until the end of the task to learn from corrections, as opposed to inferring from the corrections in an online fashion. In this paper we formalize an approach for learning from sequences of physical corrections during the current task. To do this we introduce an auxiliary reward that captures the human's trade-off between making corrections which improve the robot's immediate reward and long-term performance. We evaluate the resulting algorithm in remote and in-person human-robot experiments, and compare to both independent and final baselines. Our results indicate that users are best able to convey their objective when the robot reasons over their sequence of corrections.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。