arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2410.21153cs.CVcs.RO
收藏

Synthetica:用于机器人感知的大规模合成数据

Synthetica: Large Scale Synthetic Data for Robot Perception

NVIDIA(英伟达);University of Toronto(多伦多大学)

Ritvik Singh, Jingzhou Liu, Karl Van Wyk, Yu-Wei Chao, Jean-Francois Lafleche, Florian Shkurti, Nathan Ratliff, Ankur Handa

AI总结:

本文提出Synthetica大规模合成数据生成方法,利用照片级光线追踪渲染270万张图像及随机化、增强技术训练实时检测transformer,在目标检测上达到SOTA且速度提升9倍。

摘要:

基于视觉的目标检测器是机器人应用的关键基础,因为它们提供了有关环境中目标定位的有价值信息。这些检测器需要在不同光照条件、遮挡和视觉伪影下确保高可靠性,同时还要实时运行。为这些网络收集和标注真实世界数据极其耗时且成本高昂,尤其是对于工业对象等自定义资产,这使得其难以泛化到野外场景。为此,我们提出了 Synthetica,这是一种用于大规模合成数据生成的方法,用于训练鲁棒的状态估计器。本文聚焦于目标检测任务,这是一个重要问题,可作为大多数状态估计问题(如位姿估计)的前端。利用来自照片级真实感光线追踪渲染器的数据,我们扩大了数据生成规模,生成了270万张图像,以训练高精度的实时检测 transformer。我们提出了一系列有助于视觉任务实现鲁棒 sim-to-real 性能的渲染随机化和训练时数据增强技术。我们在目标检测任务上展示了最先进性能,同时检测器以50–100Hz运行,比此前的SOTA快9倍。我们还通过展示一个在真实世界中用于此前不存在数据集的自定义对象的流程,进一步证明了我们的训练方法对机器人应用的实用性。我们的工作强调了在实现最快实时推理速度的同时,扩大合成数据生成规模对于鲁棒 sim-to-real 迁移的重要性。视频和补充信息可在此URL找到:https://sites.google.com/view/synthetica-vision。

英文摘要:

Vision-based object detectors are a crucial basis for robotics applications as they provide valuable information about object localisation in the environment. These need to ensure high reliability in different lighting conditions, occlusions, and visual artifacts, all while running in real-time. Collecting and annotating real-world data for these networks is prohibitively time consuming and costly, especially for custom assets, such as industrial objects, making it untenable for generalization to in-the-wild scenarios. To this end, we present Synthetica, a method for large-scale synthetic data generation for training robust state estimators. This paper focuses on the task of object detection, an important problem which can serve as the front-end for most state estimation problems, such as pose estimation. Leveraging data from a photorealistic ray-tracing renderer, we scale up data generation, generating 2.7 million images, to train highly accurate real-time detection transformers. We present a collection of rendering randomization and training-time data augmentation techniques conducive to robust sim-to-real performance for vision tasks. We demonstrate state-of-the-art performance on the task of object detection while having detectors that run at 50-100Hz which is 9 times faster than the prior SOTA. We further demonstrate the usefulness of our training methodology for robotics applications by showcasing a pipeline for use in the real world with custom objects for which there do not exist prior datasets. Our work highlights the importance of scaling synthetic data generation for robust sim-to-real transfer while achieving the fastest real-time inference speeds. Videos and supplementary information can be found at this URL: https://sites.google.com/view/synthetica-vision.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑