HVDistill：通过无监督混合视图蒸馏将知识从图像迁移到点云
HVDistill: Transferring Knowledge from Images to Point Clouds via Unsupervised Hybrid-View Distillation
AI总结：
提出HVDistill框架，利用图像与点云间的混合视图（图像平面和鸟瞰）对应关系，以无监督方式将图像知识蒸馏到点云网络，无需标注，在多个数据集上显著提升下游任务性能。
摘要：
我们提出了一种基于混合视图的知识蒸馏框架，称为HVDistill，旨在以无监督的方式利用预训练的图像网络来指导点云神经网络的特征学习。通过利用RGB相机与激光雷达传感器之间的几何关系，可以基于图像平面视图和鸟瞰视图建立两种模态之间的对应关系，这有助于表示学习。具体来说，图像平面对应关系可以通过投影点云直接获得，而鸟瞰视图对应关系则可以通过在投影点云的监督下，利用预测深度将像素提升到三维空间来实现。图像教师网络从图像平面视图提供丰富的语义信息，同时从鸟瞰视图获取几何信息。实际上，来自两个视图的图像特征自然互补，共同改善点云学生网络学习到的特征表示。此外，借助自监督预训练的2D网络，HVDistill既不需要2D标注，也不需要3D标注。我们在nuScenes数据集上预训练模型，并将其迁移到nuScenes、SemanticKITTI和KITTI数据集上的多个下游任务进行评估。大量实验结果表明，我们的方法相较于从头训练的基线取得了持续改进，并显著优于现有方案。代码可在git@github.com:zhangsha1024/HVDistill.git获取。
英文摘要：
We present a hybrid-view-based knowledge distillation framework, termed HVDistill, to guide the feature learning of a point cloud neural network with a pre-trained image network in an unsupervised manner. By exploiting the geometric relationship between RGB cameras and LiDAR sensors, the correspondence between the two modalities based on both image-plane view and bird-eye view can be established, which facilitates representation learning. Specifically, the image-plane correspondences can be simply obtained by projecting the point clouds, while the bird-eye-view correspondences can be achieved by lifting pixels to the 3D space with the predicted depths under the supervision of projected point clouds. The image teacher networks provide rich semantics from the image-plane view and meanwhile acquire geometric information from the bird-eye view. Indeed, image features from the two views naturally complement each other and together can ameliorate the learned feature representation of the point cloud student networks. Moreover, with a self-supervised pre-trained 2D network, HVDistill requires neither 2D nor 3D annotations. We pre-train our model on nuScenes dataset and transfer it to several downstream tasks on nuScenes, SemanticKITTI, and KITTI datasets for evaluation. Extensive experimental results show that our method achieves consistent improvements over the baseline trained from scratch and significantly outperforms the existing schemes. Codes are available at git@github.com:zhangsha1024/HVDistill.git.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
