arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2407.20635cs.ROcs.AI
收藏

通过基础模型自主提升指令跟随技能

Autonomous Improvement of Instruction Following Skills via Foundation Models

UC Berkeley(加州大学伯克利分校)

Zhiyuan Zhou, Pranav Atreya, Abraham Lee, Homer Walke, Oier Mees, Sergey Levine

AI总结:

提出一种利用视觉语言模型和任务分解方法,使机器人从自主收集的数据中无需人类监督即可提升指令跟随能力,实验显示在未见环境中性能提升2倍。

摘要:

能够从自主收集的经验中提升的智能指令跟随机器人有潜力改变机器人学习:无需收集昂贵的遥操作演示数据,大规模部署机器人车队可以快速收集更大量的自主数据,从而集体提升其性能。然而,自主提升需要解决两个关键问题:(i)完全自动化可扩展的数据收集程序,以收集多样且语义上有意义的机器人数据;(ii)从无人类标注的非最优自主数据中学习。为此,我们提出了一种新颖的方法来解决这些挑战,使指令跟随策略能够在无需人类监督的情况下从自主收集的数据中提升。我们的框架利用视觉语言模型来收集和评估新环境中语义上有意义的经验,然后利用将指令跟随任务分解为(语义的)语言条件图像生成和(非语义的)目标到达,这使得从这些自主收集的数据中提升变得更加实用,且无需任何人类标注。我们在现实世界中进行了大量实验以证明我们方法的有效性,并发现在一组未见过的环境中,机器人策略可以通过自主收集的数据提升2倍。我们开源了我们的语义自主提升管道的代码,以及我们在五个桌面环境中收集的30.5K轨迹的自主数据集。

英文摘要:

Intelligent instruction-following robots capable of improving from autonomously collected experience have the potential to transform robot learning: instead of collecting costly teleoperated demonstration data, large-scale deployment of fleets of robots can quickly collect larger quantities of autonomous data that can collectively improve their performance. However, autonomous improvement requires solving two key problems: (i) fully automating a scalable data collection procedure that can collect diverse and semantically meaningful robot data and (ii) learning from non-optimal, autonomous data with no human annotations. To this end, we propose a novel approach that addresses these challenges, allowing instruction-following policies to improve from autonomously collected data without human supervision. Our framework leverages vision-language models to collect and evaluate semantically meaningful experiences in new environments, and then utilizes a decomposition of instruction following tasks into (semantic) language-conditioned image generation and (non-semantic) goal reaching, which makes it significantly more practical to improve from this autonomously collected data without any human annotations. We carry out extensive experiments in the real world to demonstrate the effectiveness of our approach, and find that in a suite of unseen environments, the robot policy can be improved 2x with autonomously collected data. We open-source the code for our semantic autonomous improvement pipeline, as well as our autonomous dataset of 30.5K trajectories collected across five tabletop environments.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑