arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2406.03757cs.ROcs.LG
收藏

RoboCoder:基于大语言模型从基础技能到通用任务的机器人学习

RoboCoder: Robotic Learning from Basic Skills to General Tasks with Large Language Models

CUHK(香港中文大学);SmartMore(思谋科技)

Jingyao Li, Pengguang Chen, Sitong Wu, Chuanyang Zheng, Hong Xu, Jiaya Jia

AI总结:

本文提出RoboCoder基准与自主学习方法框架,通过结合大语言模型和实时环境反馈动态更新动作代码,使机器人从基础技能泛化到复杂任务,并实现36%的相对性能提升。

摘要:

大语言模型(LLMs)的出现改善了机器人任务的前景。然而,现有基准仍局限于单一任务,泛化能力有限。在本工作中,我们提出了一个综合性基准和一个自主学习方法框架 RoboCoder,旨在增强机器人在复杂环境中的泛化能力。与传统方法专注于单任务学习不同,我们的研究强调开发一种通用机器人编码算法,使机器人能够利用基础技能来处理日益复杂的任务。新提出的基准包含跨越7类不同实体的80个人工设计任务,用于测试模型在最小初始掌握程度下的学习能力。初步测试表明,即使是GPT-4这样的先进模型,在人形实体的三样本(three-shot)场景中也仅能达到47%的通过率。为了解决这些局限,RoboCoder框架将大语言模型(LLMs)与动态学习系统相结合,利用实时环境反馈持续更新和优化动作代码。这种自适应方法表现出显著提升,实现了36%的相对改进。我们的代码将公开发布。

英文摘要:

The emergence of Large Language Models (LLMs) has improved the prospects for robotic tasks. However, existing benchmarks are still limited to single tasks with limited generalization capabilities. In this work, we introduce a comprehensive benchmark and an autonomous learning framework, RoboCoder aimed at enhancing the generalization capabilities of robots in complex environments. Unlike traditional methods that focus on single-task learning, our research emphasizes the development of a general-purpose robotic coding algorithm that enables robots to leverage basic skills to tackle increasingly complex tasks. The newly proposed benchmark consists of 80 manually designed tasks across 7 distinct entities, testing the models' ability to learn from minimal initial mastery. Initial testing revealed that even advanced models like GPT-4 could only achieve a 47% pass rate in three-shot scenarios with humanoid entities. To address these limitations, the RoboCoder framework integrates Large Language Models (LLMs) with a dynamic learning system that uses real-time environmental feedback to continuously update and refine action codes. This adaptive method showed a remarkable improvement, achieving a 36% relative improvement. Our codes will be released.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑