arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2408.05285cs.LGcs.AI
收藏

半监督单样本模仿学习

Semi-Supervised One-Shot Imitation Learning

University of California, Berkeley(加州大学伯克利分校);Google Deepmind(谷歌DeepMind)

Philipp Wu, Kourosh Hakhamaneshi, Yuqing Du, Igor Mordatch, Aravind Rajeswaran, Pieter Abbeel

AI总结:

针对单样本模仿学习依赖大量成对演示的问题,本文提出半监督 OSIL 设定,并利用嵌入聚类自生成轨迹配对,在模拟控制任务中达到接近真实标签监督的性能。

摘要:

单样本模仿学习(One-shot Imitation Learning,OSIL)旨在赋予 AI 智能体从单个演示中学习新任务的能力。为了监督学习过程,OSIL 通常需要数量多到令人难以承受的成对专家演示——即对应同一语义任务不同变体的轨迹。为克服这一限制,我们提出半监督 OSIL 问题设定:学习智能体获得一个不含任务标签的大型轨迹数据集(即非成对数据集),以及一个每个语义任务包含多个演示的小型数据集(即成对数据集)。这体现了一种更现实、更实用的少样本学习形式,并要求智能体有效利用来自大型轨迹数据集的弱监督。随后,我们开发了一种专门适用于该半监督 OSIL 设定的算法。我们的方法首先学习一个嵌入空间,使不同任务在其中形成独特聚类。我们利用该嵌入空间及其支持的聚类,在大型非成对数据集中的轨迹之间自生成配对。通过在模拟控制任务上的实验结果,我们证明,使用此类自生成配对训练的 OSIL 模型可与使用真实标签训练的 OSIL 模型相竞争,这代表了 OSIL 标签效率方面的一项重大进展。

英文摘要:

One-shot Imitation Learning~(OSIL) aims to imbue AI agents with the ability to learn a new task from a single demonstration. To supervise the learning, OSIL typically requires a prohibitively large number of paired expert demonstrations -- i.e. trajectories corresponding to different variations of the same semantic task. To overcome this limitation, we introduce the semi-supervised OSIL problem setting, where the learning agent is presented with a large dataset of trajectories with no task labels (i.e. an unpaired dataset), along with a small dataset of multiple demonstrations per semantic task (i.e. a paired dataset). This presents a more realistic and practical embodiment of few-shot learning and requires the agent to effectively leverage weak supervision from a large dataset of trajectories. Subsequently, we develop an algorithm specifically applicable to this semi-supervised OSIL setting. Our approach first learns an embedding space where different tasks cluster uniquely. We utilize this embedding space and the clustering it supports to self-generate pairings between trajectories in the large unpaired dataset. Through empirical results on simulated control tasks, we demonstrate that OSIL models trained on such self-generated pairings are competitive with OSIL models trained with ground-truth labels, presenting a major advancement in the label-efficiency of OSIL.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑