RoboEngine:基于语义机器人分割与背景生成的即插即用机器人数据增强
RoboEngine: Plug-and-Play Robot Data Augmentation with Semantic Robot Segmentation and Background Generation
AI总结:
本文提出首个即插即用的视觉机器人数据增强工具包RoboEngine,通过构建机器人场景分割数据集、高质量分割模型和微调背景生成模型,仅需几行代码即可生成物理与任务感知场景,使机器人操作任务在全新场景中实现超200%的性能提升。
摘要:
视觉增强已成为提升模仿学习视觉鲁棒性的关键技术。然而,现有方法常受限于相机校准等先决条件或对受控环境(如绿幕设置)的需求。在此项工作中,我们推出了RoboEngine,这是首个即插即用的视觉机器人数据增强工具包。用户首次能够仅用几行代码,轻松生成具备物理感知与任务感知的机器人场景。为此,我们提出了一个新颖的机器人场景分割数据集、一个可泛化的高质量机器人分割模型,以及一个微调的背景生成模型,它们共同构成了这一开箱即用工具包的核心组件。使用RoboEngine,我们展示了机器人操作任务在六个全新场景中的泛化能力,仅基于从单一场景收集的演示数据,与无增强基线相比实现了超过200%的性能提升。所有数据集、模型权重和工具包均已发布。
英文摘要:
Visual augmentation has become a crucial technique for enhancing the visual robustness of imitation learning. However, existing methods are often limited by prerequisites such as camera calibration or the need for controlled environments (e.g., green screen setups). In this work, we introduce RoboEngine, the first plug-and-play visual robot data augmentation toolkit. For the first time, users can effortlessly generate physics- and task-aware robot scenes with just a few lines of code. To achieve this, we present a novel robot scene segmentation dataset, a generalizable high-quality robot segmentation model, and a fine-tuned background generation model, which together form the core components of the out-of-the-box toolkit. Using RoboEngine, we demonstrate the ability to generalize robot manipulation tasks across six entirely new scenes, based solely on demonstrations collected from a single scene, achieving a more than 200% performance improvement compared to the no-augmentation baseline. All datasets, model weights, and the toolkit are released https://roboengine.github.io/
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。