FSD:快速自监督单张 RGB-D 图像到类别级 3D 物体
FSD: Fast Self-Supervised Single RGB-D to Categorical 3D Objects
AI总结:
本文提出 FSD 方法,通过多阶段训练管线结合 2D 监督与 3D 自监督损失,实现单张 RGB-D 图像的类别级 3D 物体形状、尺寸和 6D 位姿快速预测,在 NOCS 测试集上 mAP 提升 16.4% 并支持近实时运行。
摘要:
在本研究中,我们解决了在不依赖真实世界 3D 标注数据的情况下进行 3D 物体识别这一具有挑战性的任务。我们的目标是在单张 RGB-D 图像中预测物体的 3D 形状、尺寸和 6D 位姿,在类别级别上运行,并消除推理过程中对 CAD 模型的需求。尽管现有的自监督方法在该领域取得了进展,但它们通常存在效率低下的问题,这源于非端到端处理、针对不同物体类别依赖独立模型,以及在隐式重建模型训练期间表面提取缓慢;从而阻碍了 3D 识别过程的速度和真实世界适用性。我们提出的方法利用多阶段训练管线,旨在高效地将合成性能迁移到真实世界领域。该方法通过在合成领域训练期间结合 2D 和 3D 监督损失,随后在两个额外学习阶段对真实世界数据引入 2D 监督和 3D 自监督损失来实现。通过采用这一综合策略,我们的方法成功克服了上述局限性,并在 NOCS 测试集上超越了现有的自监督 6D 位姿和尺寸估计基线,在 6D 位姿估计的 mAP 上实现了 16.4% 的绝对提升,同时以 5 Hz 的频率近实时运行。
英文摘要:
In this work, we address the challenging task of 3D object recognition without the reliance on real-world 3D labeled data. Our goal is to predict the 3D shape, size, and 6D pose of objects within a single RGB-D image, operating at the category level and eliminating the need for CAD models during inference. While existing self-supervised methods have made strides in this field, they often suffer from inefficiencies arising from non-end-to-end processing, reliance on separate models for different object categories, and slow surface extraction during the training of implicit reconstruction models; thus hindering both the speed and real-world applicability of the 3D recognition process. Our proposed method leverages a multi-stage training pipeline, designed to efficiently transfer synthetic performance to the real-world domain. This approach is achieved through a combination of 2D and 3D supervised losses during the synthetic domain training, followed by the incorporation of 2D supervised and 3D self-supervised losses on real-world data in two additional learning stages. By adopting this comprehensive strategy, our method successfully overcomes the aforementioned limitations and outperforms existing self-supervised 6D pose and size estimation baselines on the NOCS test-set with a 16.4% absolute improvement in mAP for 6D pose estimation while running in near real-time at 5 Hz.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。