通过双重去噪从CLIP学习鲁棒的三维表示
Learning Robust 3D Representation from CLIP via Dual Denoising
AI总结：
本文提出Dual Denoising框架，通过去噪代理任务与特征去噪网络从CLIP学习鲁棒三维表示，并结合并行噪声推理提升跨域泛化，在零样本下无需对抗训练即可增强表示性能与对抗鲁棒性。
摘要：
本文探讨了一个关键但尚未被充分研究的问题：如何从预训练的视觉语言模型（如CLIP）中学习鲁棒且泛化良好的三维表示。已有工作表明，跨模态蒸馏可以为三维数据提供丰富且有用的知识。然而，与大多数深度学习模型一样，由此得到的三维学习网络仍然容易受到对抗攻击，尤其是迭代攻击。在这项工作中，我们提出了Dual Denoising，这是一个从CLIP学习鲁棒且泛化良好的三维表示的新框架。它将基于去噪的代理任务与一种新颖的特征去噪网络相结合，用于三维预训练。此外，我们提出利用并行噪声推理来增强跨域设置下点云特征的泛化能力。实验表明，我们的模型能够在零样本设置下，在不进行对抗训练的情况下，有效提升三维学习网络的表示学习性能和对抗鲁棒性。代码可在 https://github.com/luoshuqing2001/Dual_Denoising 获取。
英文摘要：
In this paper, we explore a critical yet under-investigated issue: how to learn robust and well-generalized 3D representation from pre-trained vision language models such as CLIP. Previous works have demonstrated that cross-modal distillation can provide rich and useful knowledge for 3D data. However, like most deep learning models, the resultant 3D learning network is still vulnerable to adversarial attacks especially the iterative attack. In this work, we propose Dual Denoising, a novel framework for learning robust and well-generalized 3D representations from CLIP. It combines a denoising-based proxy task with a novel feature denoising network for 3D pre-training. Additionally, we propose utilizing parallel noise inference to enhance the generalization of point cloud features under cross domain settings. Experiments show that our model can effectively improve the representation learning performance and adversarial robustness of the 3D learning network under zero-shot settings without adversarial training. Our code is available at https://github.com/luoshuqing2001/Dual_Denoising.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
