通过从2D Transformer蒸馏关系先验来提升跨域点云分类
Boosting Cross-Domain Point Classification via Distilling Relational Priors from 2D Transformers
AI总结：
针对点云分类跨域泛化难题，提出RPD方法，从2D预训练Transformer向3D学生蒸馏拓扑关系先验，并结合多视图掩码重建自监督任务，在PointDA-10和Sim-to-Real上达到SOTA。
摘要：
物体点云的语义模式由其局部几何的拓扑构型决定。由于局部区域中点集形状变化巨大，且从全局视角看表面不完整，学习具有判别力的表征可能颇具挑战；在无监督域适应（UDA）场景下，这一问题会更加严重。具体而言，传统3D网络主要关注局部几何细节，而忽视局部几何之间的拓扑结构，这极大限制了它们的跨域泛化能力。近来，基于Transformer的模型在一系列基于图像的任务中取得了令人瞩目的性能提升，这得益于其通过捕获跨局部块的长程相关性而产生的强泛化能力和可扩展性。受视觉Transformer此类成功的启发，我们提出了一种新颖的关系先验蒸馏（RPD）方法，用于从在海量图像上训练良好的Transformer中提取关系先验，从而以一致的物体拓扑先验显著增强跨域表征。为此，我们建立了一个在2D教师模型与3D学生模型之间共享的参数冻结预训练Transformer模块，并辅以在线知识蒸馏策略，对3D学生模型进行语义正则化。此外，我们引入了一项新颖的自监督任务，其核心是利用相应的被遮挡多视图图像特征重建被遮挡的点云块，从而使模型能够融合3D几何信息。在PointDA-10和Sim-to-Real数据集上的实验验证了，所提方法在点云分类的UDA中持续取得最先进性能。本工作的源代码可在https://github.com/zou-longkun/RPD.git获取。
英文摘要：
Semantic pattern of an object point cloud is determined by its topological configuration of local geometries. Learning discriminative representations can be challenging due to large shape variations of point sets in local regions and incomplete surface in a global perspective, which can be made even more severe in the context of unsupervised domain adaptation (UDA). In specific, traditional 3D networks mainly focus on local geometric details and ignore the topological structure between local geometries, which greatly limits their cross-domain generalization. Recently, the transformer-based models have achieved impressive performance gain in a range of image-based tasks, benefiting from its strong generalization capability and scalability stemming from capturing long range correlation across local patches. Inspired by such successes of visual transformers, we propose a novel Relational Priors Distillation (RPD) method to extract relational priors from the well-trained transformers on massive images, which can significantly empower cross-domain representations with consistent topological priors of objects. To this end, we establish a parameter-frozen pre-trained transformer module shared between 2D teacher and 3D student models, complemented by an online knowledge distillation strategy for semantically regularizing the 3D student model. Furthermore, we introduce a novel self-supervised task centered on reconstructing masked point cloud patches using corresponding masked multi-view image features, thereby empowering the model with incorporating 3D geometric information. Experiments on the PointDA-10 and the Sim-to-Real datasets verify that the proposed method consistently achieves the state-of-the-art performance of UDA for point cloud classification. The source code of this work is available at https://github.com/zou-longkun/RPD.git.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
