先预训练模型然后在下游任务上进行微调,在2D图像和NLP领域已经取得了显著的成功。然而,由于点云的无序和非均匀密度特性,探索点云的先验知识并预训练点云骨干网络并非易事。在本文中,我们提出了一种名为点云扩散预训练(PointDif)的新型预训练方法。我们将点云预训练任务视为一个条件点对点生成问题,并引入了一个条件点生成器。该生成器聚合由骨干网络提取的特征,并将其作为条件,指导从噪声点云中进行点对点恢复,从而协助骨干网络捕获局部和全局几何先验以及物体的全局点密度分布。我们还提出了一种循环均匀采样优化策略,使模型能够从各种噪声水平中均匀恢复,并从均衡的监督中学习。我们的PointDif在多个真实世界数据集上的各种下游任务(如分类、分割和检测)中取得了显著改进。具体而言,PointDif在S3DIS Area 5的分割任务中达到了70.0%的mIoU,并且在ScanObjectNN的分类任务上与TAP相比实现了平均2.4%的提升。此外,我们的预训练框架可以灵活地应用于各种点云骨干网络,并带来可观的收益。
英文摘要:
Pre-training a model and then fine-tuning it on downstream tasks has demonstrated significant success in the 2D image and NLP domains. However, due to the unordered and non-uniform density characteristics of point clouds, it is non-trivial to explore the prior knowledge of point clouds and pre-train a point cloud backbone. In this paper, we propose a novel pre-training method called Point cloud Diffusion pre-training (PointDif). We consider the point cloud pre-training task as a conditional point-to-point generation problem and introduce a conditional point generator. This generator aggregates the features extracted by the backbone and employs them as the condition to guide the point-to-point recovery from the noisy point cloud, thereby assisting the backbone in capturing both local and global geometric priors as well as the global point density distribution of the object. We also present a recurrent uniform sampling optimization strategy, which enables the model to uniformly recover from various noise levels and learn from balanced supervision. Our PointDif achieves substantial improvement across various real-world datasets for diverse downstream tasks such as classification, segmentation and detection. Specifically, PointDif attains 70.0% mIoU on S3DIS Area 5 for the segmentation task and achieves an average improvement of 2.4% on ScanObjectNN for the classification task compared to TAP. Furthermore, our pre-training framework can be flexibly applied to diverse point cloud backbones and bring considerable gains.
SADER: Structure-Aware Diffusion Framework with DEterministic Resampling for Multi-Temporal Remote Sensing Cloud Removal
College of Literature, Science, and the Arts, University of Michigan(文学、科学与艺术学院,密歇根大学);School of Computer Science and Technology, Tongji University(计算机科学与技术学院,同济大学)
Harnessing Text-to-Image Diffusion Models for Point Cloud Self-Supervised Learning
South China University of Technology(华南理工大学);Alibaba International Digital Commerce Group(阿里巴巴国际数字商务集团);Pazhou Lab(琶洲实验室);Nanyang Technological University(南洋理工大学)
A Novel Cloud-Based Diffusion-Guided Hybrid Model for High-Accuracy Accident Detection in Intelligent Transportation Systems
Department of Electrical and Electronics Engineering, BITS-Pilani, Pilani Campus(比特学院电子与电子工程系);The Quantum Cloud Computing and Distributed Systems (qCLOUDS) Laboratory, School of Computing and Information Systems, The University of Melbourne(墨尔本大学计算机与信息系统学院量子云计算与分布式系统实验室);qCLOUDS Laboratory(qCLOUDS实验室)
IDF-CR: Iterative Diffusion Process for Divide-and-Conquer Cloud Removal in Remote-sensing Images
School of Information Engineering, Guangdong University of Technology(广东工业大学信息工程学院);CRRC Academy Co., Ltd.(中车研究院有限公司);School of Computer Science, Sun Yat-sen University(中山大学计算机学院)
Diffusion Enhancement for Cloud Removal in Ultra-Resolution Remote Sensing Imagery
The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳));Peking University(北京大学);Peng Cheng Laboratory(鹏城实验室);Wuhan University of Science and Technology(武汉科技大学)
Recovering Cloud Microstructures with Cascaded Diffusion Inversion
Mohamed Bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学);Hebrew University of Jerusalem(耶路撒冷希伯来大学);University of California, San Diego(加利福尼亚大学圣地亚哥分校)
PointDiffuse: A Dual-Conditional Diffusion Model for Enhanced Point Cloud Semantic Segmentation
Anhui University(安徽大学);Hunan University(湖南大学);Xidian University(西安电子科技大学);University of Western Australia(西澳大学);Central Queensland University(中央昆士兰大学);Australian National University(澳大利亚国立大学)
Enhancing Local Feature Learning Using Diffusion for 3D Point Cloud Understanding
Tokyo Institute of Technology(东京工业大学);Artificial Intelligence Research Center, AIST(产业技术综合研究所人工智能研究中心);Dalian University of Technology(大连理工大学);PASCO Corporation(PASCO公司)