arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2310.12974cs.CVcs.RO
收藏

FSD:快速自监督单张 RGB-D 图像到类别级 3D 物体

FSD: Fast Self-Supervised Single RGB-D to Categorical 3D Objects

Georgia Institute of Technology(佐治亚理工学院);Toyota Research Institute(丰田研究院)

Mayank Lunayach, Sergey Zakharov, Dian Chen, Rares Ambrus, Zsolt Kira, Muhammad Zubair Irshad

AI总结:

本文提出 FSD 方法,通过多阶段训练管线结合 2D 监督与 3D 自监督损失,实现单张 RGB-D 图像的类别级 3D 物体形状、尺寸和 6D 位姿快速预测,在 NOCS 测试集上 mAP 提升 16.4% 并支持近实时运行。

摘要:

在本研究中,我们解决了在不依赖真实世界 3D 标注数据的情况下进行 3D 物体识别这一具有挑战性的任务。我们的目标是在单张 RGB-D 图像中预测物体的 3D 形状、尺寸和 6D 位姿,在类别级别上运行,并消除推理过程中对 CAD 模型的需求。尽管现有的自监督方法在该领域取得了进展,但它们通常存在效率低下的问题,这源于非端到端处理、针对不同物体类别依赖独立模型,以及在隐式重建模型训练期间表面提取缓慢;从而阻碍了 3D 识别过程的速度和真实世界适用性。我们提出的方法利用多阶段训练管线,旨在高效地将合成性能迁移到真实世界领域。该方法通过在合成领域训练期间结合 2D 和 3D 监督损失,随后在两个额外学习阶段对真实世界数据引入 2D 监督和 3D 自监督损失来实现。通过采用这一综合策略,我们的方法成功克服了上述局限性,并在 NOCS 测试集上超越了现有的自监督 6D 位姿和尺寸估计基线,在 6D 位姿估计的 mAP 上实现了 16.4% 的绝对提升,同时以 5 Hz 的频率近实时运行。

英文摘要:

In this work, we address the challenging task of 3D object recognition without the reliance on real-world 3D labeled data. Our goal is to predict the 3D shape, size, and 6D pose of objects within a single RGB-D image, operating at the category level and eliminating the need for CAD models during inference. While existing self-supervised methods have made strides in this field, they often suffer from inefficiencies arising from non-end-to-end processing, reliance on separate models for different object categories, and slow surface extraction during the training of implicit reconstruction models; thus hindering both the speed and real-world applicability of the 3D recognition process. Our proposed method leverages a multi-stage training pipeline, designed to efficiently transfer synthetic performance to the real-world domain. This approach is achieved through a combination of 2D and 3D supervised losses during the synthetic domain training, followed by the incorporation of 2D supervised and 3D self-supervised losses on real-world data in two additional learning stages. By adopting this comprehensive strategy, our method successfully overcomes the aforementioned limitations and outperforms existing self-supervised 6D pose and size estimation baselines on the NOCS test-set with a 16.4% absolute improvement in mAP for 6D pose estimation while running in near real-time at 5 Hz.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑