行为检索：通过查询未标注数据集进行少样本模仿学习
Behavior Retrieval: Few-Shot Imitation Learning by Querying Unlabeled Datasets
AI总结：
提出行为检索方法，利用少量专家数据从未标注数据集中选择性查询相关行为，联合训练提升少样本模仿学习效果，在模拟和真实机器人任务中优于复杂目标条件方法20%。
摘要：
使机器人以数据高效的方式学习新的视觉运动技能仍然是一个未解决且充满挑战的问题。解决该问题的一种流行范式是利用包含许多行为的大型未标注数据集，然后使用少量特定任务的人类监督（即干预或演示）将策略调整到特定任务。然而，如何最好地利用狭窄的特定任务监督并平衡离线数据仍是一个开放性问题。我们在这项工作中的关键见解是，特定任务数据不仅为智能体提供新的训练数据，还可以告知智能体应使用何种类型的先验数据进行学习。具体而言，我们提出了一种简单方法，使用少量下游专家数据从离线未标注数据集（包括许多次优行为）中选择性地查询相关行为。智能体随后在专家数据和查询数据上联合训练。我们观察到，我们的方法学会仅查询与任务相关的转换，过滤掉次优或任务无关的数据。通过这样做，与朴素混合数据或仅使用特定任务数据相比，它能够更有效地从特定任务和离线数据的混合中学习。此外，我们发现，我们简单的查询方法在图像上的模拟和真实机器人操作任务中，比更复杂的目标条件方法平均高出20%。视频和代码见 https://sites.google.com/view/behaviorretrieval。
英文摘要：
Enabling robots to learn novel visuomotor skills in a data-efficient manner remains an unsolved problem with myriad challenges. A popular paradigm for tackling this problem is through leveraging large unlabeled datasets that have many behaviors in them and then adapting a policy to a specific task using a small amount of task-specific human supervision (i.e. interventions or demonstrations). However, how best to leverage the narrow task-specific supervision and balance it with offline data remains an open question. Our key insight in this work is that task-specific data not only provides new data for an agent to train on but can also inform the type of prior data the agent should use for learning. Concretely, we propose a simple approach that uses a small amount of downstream expert data to selectively query relevant behaviors from an offline, unlabeled dataset (including many sub-optimal behaviors). The agent is then jointly trained on the expert and queried data. We observe that our method learns to query only the relevant transitions to the task, filtering out sub-optimal or task-irrelevant data. By doing so, it is able to learn more effectively from the mix of task-specific and offline data compared to naively mixing the data or only using the task-specific data. Furthermore, we find that our simple querying approach outperforms more complex goal-conditioned methods by 20% across simulated and real robotic manipulation tasks from images. See https://sites.google.com/view/behaviorretrieval for videos and code.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
