arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2311.14960cs.CV
收藏

使用扩散模型进行点云预训练

Point Cloud Pre-training with Diffusion Models

Shandong University(山东大学);Shanghai AI Laboratory(上海人工智能实验室);Fondazione Bruno Kessler(布鲁诺·凯斯勒基金会)

Xiao Zheng, Xiaoshui Huang, Guofeng Mei, Yuenan Hou, Zhaoyang Lyu, Bo Dai, Wanli Ouyang, Yongshun Gong

AI总结:

本文提出一种新型点云预训练方法PointDif,将预训练视为条件点对点生成问题,通过条件点生成器引导从噪声点云中恢复点云,帮助骨干网络捕获几何先验和密度分布,并结合循环均匀采样优化策略,在分类、分割和检测等下游任务上取得显著提升。

摘要:

先预训练模型然后在下游任务上进行微调,在2D图像和NLP领域已经取得了显著的成功。然而,由于点云的无序和非均匀密度特性,探索点云的先验知识并预训练点云骨干网络并非易事。在本文中,我们提出了一种名为点云扩散预训练(PointDif)的新型预训练方法。我们将点云预训练任务视为一个条件点对点生成问题,并引入了一个条件点生成器。该生成器聚合由骨干网络提取的特征,并将其作为条件,指导从噪声点云中进行点对点恢复,从而协助骨干网络捕获局部和全局几何先验以及物体的全局点密度分布。我们还提出了一种循环均匀采样优化策略,使模型能够从各种噪声水平中均匀恢复,并从均衡的监督中学习。我们的PointDif在多个真实世界数据集上的各种下游任务(如分类、分割和检测)中取得了显著改进。具体而言,PointDif在S3DIS Area 5的分割任务中达到了70.0%的mIoU,并且在ScanObjectNN的分类任务上与TAP相比实现了平均2.4%的提升。此外,我们的预训练框架可以灵活地应用于各种点云骨干网络,并带来可观的收益。

英文摘要:

Pre-training a model and then fine-tuning it on downstream tasks has demonstrated significant success in the 2D image and NLP domains. However, due to the unordered and non-uniform density characteristics of point clouds, it is non-trivial to explore the prior knowledge of point clouds and pre-train a point cloud backbone. In this paper, we propose a novel pre-training method called Point cloud Diffusion pre-training (PointDif). We consider the point cloud pre-training task as a conditional point-to-point generation problem and introduce a conditional point generator. This generator aggregates the features extracted by the backbone and employs them as the condition to guide the point-to-point recovery from the noisy point cloud, thereby assisting the backbone in capturing both local and global geometric priors as well as the global point density distribution of the object. We also present a recurrent uniform sampling optimization strategy, which enables the model to uniformly recover from various noise levels and learn from balanced supervision. Our PointDif achieves substantial improvement across various real-world datasets for diverse downstream tasks such as classification, segmentation and detection. Specifically, PointDif attains 70.0% mIoU on S3DIS Area 5 for the segmentation task and achieves an average improvement of 2.4% on ScanObjectNN for the classification task compared to TAP. Furthermore, our pre-training framework can be flexibly applied to diverse point cloud backbones and bring considerable gains.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑