arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2307.15569cs.CV
收藏

点云是特殊图像:一种用于3D理解的知识迁移方法

Point Clouds Are Specialized Images: A Knowledge Transfer Approach for 3D Understanding

School of Electrical and Data Engineering, University of Technology Sydney(悉尼科技大学电气与数据工程学院);School of Computer Science, University of Nottingham Ningbo(诺丁汉大学宁波校区计算机学院);Department of Electrical and Electronic Engineering, The Hong Kong Polytechnic University(香港理工大学电机及电子工程系)

Jiachen Kang, Wenjing Jia, Xiangjian He, Kin Man Lam

AI总结:

本文提出PCExpert,将点云视为“特殊图像”,通过多路Transformer架构与预训练图像编码器共享参数并采用变换估计前置任务,实现高效3D自监督表示学习,显著减少可训练参数并在多项任务上超越现有技术。

摘要:

自监督表示学习(SSRL)在点云理解中受到越来越多的关注,以应对3D数据稀缺和高标注成本带来的挑战。本文提出了PCExpert,一种新颖的SSRL方法,将点云重新解释为“特殊图像”。这种概念转变使PCExpert能够通过在多路Transformer架构中与预训练图像编码器广泛共享参数,以更直接和更深入的方式利用从大规模图像模态中衍生出的知识。参数共享策略结合一种新颖的预训练前置任务,即变换估计,使PCExpert在各种任务中超越了现有技术,同时显著减少了可训练参数的数量。值得注意的是,PCExpert在LINEAR微调下的性能(例如,在ScanObjectNN上产生90.02%的总体准确率)已经接近FULL模型微调获得的结果(92.66%),证明了其有效且鲁棒的表示能力。

英文摘要:

Self-supervised representation learning (SSRL) has gained increasing attention in point cloud understanding, in addressing the challenges posed by 3D data scarcity and high annotation costs. This paper presents PCExpert, a novel SSRL approach that reinterprets point clouds as "specialized images". This conceptual shift allows PCExpert to leverage knowledge derived from large-scale image modality in a more direct and deeper manner, via extensively sharing the parameters with a pre-trained image encoder in a multi-way Transformer architecture. The parameter sharing strategy, combined with a novel pretext task for pre-training, i.e., transformation estimation, empowers PCExpert to outperform the state of the arts in a variety of tasks, with a remarkable reduction in the number of trainable parameters. Notably, PCExpert's performance under LINEAR fine-tuning (e.g., yielding a 90.02% overall accuracy on ScanObjectNN) has already approached the results obtained with FULL model fine-tuning (92.66%), demonstrating its effective and robust representation capability.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑