基于对比掩码自编码器的自监督哈希方法用于2D图像与3D点云跨模态检索
Contrastive masked auto-encoders based self-supervised hashing for 2D image and 3D point cloud cross-modal retrieval
AI总结：
针对2D图像与3D点云跨模态检索中现有方法语义捕捉不足、模态鸿沟难弥合的问题，提出基于对比掩码自编码器的自监督哈希方法CMAH，通过对比学习、多模态自编码器及融合模块提升性能，在三个公开数据集上优于所有基线。
摘要：
在现实世界的检索系统中，实现2D图像与3D点云数据之间的跨模态哈希是一个日益受关注的问题。直接将现有跨模态方法应用于这一新任务，无法充分捕捉潜在的多模态语义，也无法有效弥合2D与3D之间的模态鸿沟。为在不依赖人工标注标签的情况下解决这些问题，我们提出了基于对比掩码自编码器的自监督哈希方法（CMAH），用于图像与点云数据之间的检索。我们首先对2D-3D样本对进行对比学习，将其显式约束到联合汉明空间中。该对比学习过程确保生成的哈希码具备鲁棒的判别性，有效缩小模态鸿沟。此外，我们利用多模态自编码器增强模型对多模态语义的理解，通过完成掩码图像/点云数据的建模任务，促使模型捕捉更多局部线索。同时，所提出的多模态融合模块促进了不同模态之间的细粒度交互。在三个公开数据集上的大量实验表明，所提出的CMAH方法性能显著优于所有基线方法。
英文摘要：
Implementing cross-modal hashing between 2D images and 3D point-cloud data is a growing concern in real-world retrieval systems. Simply applying existing cross-modal approaches to this new task fails to adequately capture latent multi-modal semantics and effectively bridge the modality gap between 2D and 3D. To address these issues without relying on hand-crafted labels, we propose contrastive masked autoencoders based self-supervised hashing (CMAH) for retrieval between images and point-cloud data. We start by contrasting 2D-3D pairs and explicitly constraining them into a joint Hamming space. This contrastive learning process ensures robust discriminability for the generated hash codes and effectively reduces the modality gap. Moreover, we utilize multi-modal auto-encoders to enhance the model's understanding of multi-modal semantics. By completing the masked image/point-cloud data modeling task, the model is encouraged to capture more localized clues. In addition, the proposed multi-modal fusion block facilitates fine-grained interactions among different modalities. Extensive experiments on three public datasets demonstrate that the proposed CMAH significantly outperforms all baseline methods.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
