arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2403.19622cs.ROcs.CV
收藏

RH20T-P:面向可组合泛化智能体的原语级机器人数据集

RH20T-P: A Primitive-Level Robotic Dataset Towards Composable Generalization Agents

Shanghai AI Laboratory(上海人工智能实验室);School of Software, Beihang University(北京航空航天大学软件学院);Shanghai Jiao Tong University(上海交通大学);University of Sydney(悉尼大学)

Zeren Chen, Zhelun Shi, Xiaoya Lu, Lehan He, Sucheng Qian, Zhenfei Yin, Wanli Ouyang, Jing Shao, Yu Qiao, Cewu Lu, Lu Sheng

AI总结:

本文提出RH20T-P原语级机器人操作数据集,包含约38k视频片段覆盖67个任务,并标准化规划-执行CGA范式,通过RA-P基线验证其能赋予智能体可组合泛化能力。

摘要:

在解决分布外任务时实现泛化能力是学习机器人操作的最终目标之一。视觉语言模型(VLMs)的最新进展表明,基于VLM的任务规划器可以通过将复合任务分解为顺序执行已掌握的原语级技能计划,来缓解解决新任务的困难。以可组合泛化智能体(CGAs)的形式,将这种可组合泛化能力适配到机器人操作中也是很有前景的。然而,社区缺乏可靠的原语技能设计和充足的原语级数据标注。因此,我们提出了RH20T-P,一个原语级机器人操作数据集,包含约38k个视频片段,覆盖真实世界场景中的67个多样化操作任务。每个片段都根据一组精心设计的、在机器人操作中常见的原语技能进行手动标注。此外,我们标准化了规划-执行CGA范式,并在我们的RH20T-P上实现了一个名为RA-P的示例基线,其在解决未见任务上的积极表现验证了所提出的数据集能够为机器人操作智能体提供可组合泛化能力。

英文摘要:

Achieving generalizability in solving out-of-distribution tasks is one of the ultimate goals of learning robotic manipulation. Recent progress of Vision-Language Models (VLMs) has shown that VLM-based task planners can alleviate the difficulty of solving novel tasks, by decomposing the compounded tasks as a plan of sequentially executing primitive-level skills that have been already mastered. It is also promising for robotic manipulation to adapt such composable generalization ability, in the form of composable generalization agents (CGAs). However, the community lacks of reliable design of primitive skills and a sufficient amount of primitive-level data annotations. Therefore, we propose RH20T-P, a primitive-level robotic manipulation dataset, which contains about 38k video clips covering 67 diverse manipulation tasks in real-world scenarios. Each clip is manually annotated according to a set of meticulously designed primitive skills that are common in robotic manipulation. Furthermore, we standardize a plan-execute CGA paradigm and implement an exemplar baseline called RA-P on our RH20T-P, whose positive performance on solving unseen tasks validates that the proposed dataset can offer composable generalization ability to robotic manipulation agents.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑