面向可泛化机器人学习的语义可控增强方法
Semantically Controllable Augmentations for Generalizable Robot Learning
AI总结：
本文提出利用预训练图像-文本生成模型对机器人数据进行语义可控增强，以低成本扩充数据集并引入丰富变体，从而提升机器人在未见真实世界环境中的操作泛化能力。
摘要：
机器人操作要泛化到未见过的真实世界场景，需要在训练期间接触多样化的数据集。然而，由于运营成本高昂，收集大规模真实世界数据集难以实现。为了让机器人学习在这些挑战下仍能实现泛化，必须利用机器人直接经验之外的数据来源或先验。在这项工作中，我们认为在大规模网络抓取数据上预训练的图像-文本生成模型可以作为这样的数据来源。这些生成模型涵盖了机器人直接经验之外的广泛真实世界场景，并且能够合成新颖的合成经验，使机器人智能体接触到额外的世界先验，从而在不增加额外成本的情况下帮助实现真实世界泛化。具体而言，我们的方法利用预训练生成模型作为数据增强的有效工具。我们提出了一个生成式增强框架，用于语义可控的增强，并快速扩充机器人数据集，同时引入丰富的变体，从而实现真实世界泛化。基于机器人数据的多样化增强，我们展示了可扩展的机器人操作策略如何在仿真以及未见过的真实世界环境（如厨房和桌面）中进行训练和部署。通过在多种真实世界机器人应用中展示图像-文本生成模型的有效性，我们的生成式增强框架为在不增加额外人力成本的情况下提升机器人学习的泛化能力提供了一条可扩展且高效的路径。
英文摘要：
Generalization to unseen real-world scenarios for robot manipulation requires exposure to diverse datasets during training. However, collecting large real-world datasets is intractable due to high operational costs. For robot learning to generalize despite these challenges, it is essential to leverage sources of data or priors beyond the robot's direct experience. In this work, we posit that image-text generative models, which are pre-trained on large corpora of web-scraped data, can serve as such a data source. These generative models encompass a broad range of real-world scenarios beyond a robot's direct experience and can synthesize novel synthetic experiences that expose robotic agents to additional world priors aiding real-world generalization at no extra cost. In particular, our approach leverages pre-trained generative models as an effective tool for data augmentation. We propose a generative augmentation framework for semantically controllable augmentations and rapidly multiplying robot datasets while inducing rich variations that enable real-world generalization. Based on diverse augmentations of robot data, we show how scalable robot manipulation policies can be trained and deployed both in simulation and in unseen real-world environments such as kitchens and table-tops. By demonstrating the effectiveness of image-text generative models in diverse real-world robotic applications, our generative augmentation framework provides a scalable and efficient path for boosting generalization in robot learning at no extra human cost.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
