结合演示与语言指令高效学习机器人任务
Using Both Demonstrations and Language Instructions to Efficiently Learn Robotic Tasks
AI总结：
针对单模态任务指定存在歧义的问题，提出DeL-TaCo方法，结合视觉演示与语言嵌入对多任务机器人策略做条件化，既降低施教成本，又提升新场景下的样本效率与泛化性能。
摘要：
演示和自然语言指令是向机器人指定并传授新任务的两种常用方式。但对于许多复杂任务，单独使用演示或语言指令都存在歧义，无法清晰地指定任务。在这类场景下，将演示与指令结合使用，比单独使用任一模态都能更简洁有效地向机器人传达任务信息。为实现这一问题设定，我们在数百项具有挑战性的机器人抓取放置任务上训练了单个多任务策略，并提出DeL-TaCo（联合演示-语言任务条件化）方法，该方法可让机器人策略基于由视觉演示和语言指令两部分组成的任务嵌入进行条件化。通过让两种模态在新任务指定过程中相互消解歧义、彼此澄清，DeL-TaCo（1）大幅降低了指定新任务所需的施教者工作量；（2）在新物体和新指令上的泛化性能优于以往的任务条件化方法。据我们所知，这是首个证明同时基于演示和语言嵌入对多任务机器人操作策略进行条件化，相比仅基于单模态条件化能提升样本效率与泛化能力的研究。更多材料请见https://deltaco-robot.github.io/
英文摘要：
Demonstrations and natural language instructions are two common ways to specify and teach robots novel tasks. However, for many complex tasks, a demonstration or language instruction alone contains ambiguities, preventing tasks from being specified clearly. In such cases, a combination of both a demonstration and an instruction more concisely and effectively conveys the task to the robot than either modality alone. To instantiate this problem setting, we train a single multi-task policy on a few hundred challenging robotic pick-and-place tasks and propose DeL-TaCo (Joint Demo-Language Task Conditioning), a method for conditioning a robotic policy on task embeddings comprised of two components: a visual demonstration and a language instruction. By allowing these two modalities to mutually disambiguate and clarify each other during novel task specification, DeL-TaCo (1) substantially decreases the teacher effort needed to specify a new task and (2) achieves better generalization performance on novel objects and instructions over previous task-conditioning methods. To our knowledge, this is the first work to show that simultaneously conditioning a multi-task robotic manipulation policy on both demonstration and language embeddings improves sample efficiency and generalization over conditioning on either modality alone. See additional materials at https://deltaco-robot.github.io/
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
