arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2207.04997cs.CV
收藏

近距离观察3D视觉自监督预训练中的不变性

A Closer Look at Invariances in Self-supervised Pre-training for 3D Vision

Institute of Industrial Information Technology, Karlsruhe Institute of Technology(卡尔斯鲁厄理工学院工业信息技术研究所)

Lanxiao Li, Michael Heizmann

摘要:

近年来,三维视觉的自监督预训练引起了越来越多的研究兴趣。为了学习信息表示,许多先前的工作利用3D特征的不变性,例如,同一场景的视图之间的透视不变性,深度和RGB图像之间的模态不变性,点云和体素之间的格式不变性。虽然他们已经取得了可喜的成果,以往的研究缺乏系统和公平的比较这些不变性。为了解决这个问题,我们的工作,第一次,引入了一个统一的框架,在此框架下,各种预训练方法可以进行调查。我们进行了大量的实验,并仔细研究了3D预训练中不同不变性的贡献。此外,我们提出了一种简单但有效的方法,使用对比学习联合预训练3D编码器和深度图编码器。使用我们的方法预训练的模型在下游任务中获得了显着的性能提升。例如,预先训练的VoteNet在SUN RGB-D和ScanNet目标检测基准测试中的表现优于以前的方法,并且具有明显的优势。

英文摘要:

Self-supervised pre-training for 3D vision has drawn increasing research interest in recent years. In order to learn informative representations, a lot of previous works exploit invariances of 3D features, e.g., perspective-invariance between views of the same scene, modality-invariance between depth and RGB images, format-invariance between point clouds and voxels. Although they have achieved promising results, previous researches lack a systematic and fair comparison of these invariances. To address this issue, our work, for the first time, introduces a unified framework, under which various pre-training methods can be investigated. We conduct extensive experiments and provide a closer look at the contributions of different invariances in 3D pre-training. Also, we propose a simple but effective method that jointly pre-trains a 3D encoder and a depth map encoder using contrastive learning. Models pre-trained with our method gain significant performance boost in downstream tasks. For instance, a pre-trained VoteNet outperforms previous methods on SUN RGB-D and ScanNet object detection benchmarks with a clear margin.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑