arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2308.04725cs.CVcs.IR
收藏

基于Transformer及其自蒸馏的旋转不变三维点集特征自监督学习

Self-supervised Learning of Rotation-invariant 3D Point Set Features using Transformer and its Self-distillation

University of Yamanashi(山梨大学);Hangzhou Dianzi University(杭州电子科技大学)

Takahiko Furuya, Zhoujie Chen, Ryutarou Ohbuchi, Zhenzhong Kuang

AI总结:

本文提出一种基于Transformer和自蒸馏的自监督学习框架,从无标注三维点集中学习旋转不变特征,通过令牌分解和自注意力聚合,结合多裁剪与剪切混合增强,实验证明其优于现有方法。

摘要:

对三维物体旋转的不变性是分析三维点集数据的一个重要性质。传统的具有旋转不变性的三维点集深度神经网络通常通过使用带标签的三维点集作为训练样本进行监督学习来获得准确的三维形状特征。然而,由于三维点集数据的快速增长和标注的高成本,需要一种从大量未标注的三维点集中学习旋转不变三维形状特征的框架。本文提出了一种新颖的自监督学习框架,用于在物体级别获取准确且旋转不变的三维点集特征。我们提出的轻量级深度神经网络架构将输入的三维点集分解为多个全局尺度区域,称为令牌(tokens),这些令牌保留了构成三维物体的部分形状的空间布局。我们采用自注意力机制来细化令牌,并将它们聚合成每个三维点集的具有表达力的旋转不变特征。我们的深度神经网络通过使用自蒸馏框架生成的伪标签进行有效训练。为了促进准确特征的学习,我们提出结合多裁剪(multi-crop)和剪切混合(cut-mix)数据增强技术来多样化用于训练的三维点集。通过全面评估,我们实证表明:(1)为监督学习设计的现有旋转不变深度神经网络架构在自监督学习场景下不一定能学习到准确的三维形状特征;(2)我们提出的算法学习到的旋转不变三维点集特征比现有算法学习到的更准确。代码可在 https://github.com/takahikof/RIPT_SDMM 获取。

英文摘要:

Invariance against rotations of 3D objects is an important property in analyzing 3D point set data. Conventional 3D point set DNNs having rotation invariance typically obtain accurate 3D shape features via supervised learning by using labeled 3D point sets as training samples. However, due to the rapid increase in 3D point set data and the high cost of labeling, a framework to learn rotation-invariant 3D shape features from numerous unlabeled 3D point sets is required. This paper proposes a novel self-supervised learning framework for acquiring accurate and rotation-invariant 3D point set features at object-level. Our proposed lightweight DNN architecture decomposes an input 3D point set into multiple global-scale regions, called tokens, that preserve the spatial layout of partial shapes composing the 3D object. We employ a self-attention mechanism to refine the tokens and aggregate them into an expressive rotation-invariant feature per 3D point set. Our DNN is effectively trained by using pseudo-labels generated by a self-distillation framework. To facilitate the learning of accurate features, we propose to combine multi-crop and cut-mix data augmentation techniques to diversify 3D point sets for training. Through a comprehensive evaluation, we empirically demonstrate that, (1) existing rotation-invariant DNN architectures designed for supervised learning do not necessarily learn accurate 3D shape features under a self-supervised learning scenario, and (2) our proposed algorithm learns rotation-invariant 3D point set features that are more accurate than those learned by existing algorithms. Code is available at https://github.com/takahikof/RIPT_SDMM

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑