arXiv 2605.309682026-05-29cs.CVcs.AI

变分适配器用于跨模态相似性表示

Variational Adapter for Cross-modal Similarity Representation

School of Remote Sensing and Information Engineering（遥感与信息工程学院）；Wuhan University（武汉大学）；School of Data Science and Engineering（数据科学与工程学院）；East China Normal University（华东师范大学）；State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing（测绘遥感信息工程国家重点实验室）

WenZhang Wei, Zhipeng Gui, Dehua Peng, Tiandi Ye, Huayi Wu

备注：: Accepted by the 43rd International Conference on Machine Learning (ICML 2026)

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对跨模态匹配中细粒度标注稀缺导致二元分类边界压缩和假负样本问题，提出变分适配器VACSR，将匹配任务重构为变分推断问题，通过构建潜在相似性空间和正则化缓解过拟合，在图像-文本检索、域泛化和基类到新类泛化任务上验证了有效性。

摘要：

视觉-语言模型的核心在于在统一表示空间中度量跨模态相似性。然而，大多数图像-文本匹配或多类图像分类数据集缺乏细粒度的跨模态匹配标注，迫使连续的相似性空间压缩为二元分类边界。这种压缩引入了假负样本，并严重损害了跨模态任务的泛化性能。尽管先前的研究试图通过建模模态内模糊性来缓解这一问题，但往往忽略了固有的标注缺陷，导致不确定性分配次优。为了解决这些挑战，我们提出了一种变分适配器用于跨模态相似性表示（VACSR）。该方法将具有细粒度语义稀缺性的图像-文本匹配重新表述为变分推断问题。它构建了一个跨模态相似性的潜在空间，并使用正则化技术来减轻对二元标注的过拟合。在图像-文本检索、域泛化和基类到新类泛化上的实验证明了所提出方法的有效性和鲁棒的泛化能力。

英文摘要：

The core of vision-language models lies in measuring cross-modal similarity within a unified representation space. However, most image-text matching or multi-class image classification datasets lack fine-grained cross-modal matching annotations, forcing the continuous similarity space into binary classification boundaries. This compression induces false negative samples and significantly impairs the generalization performance of cross-modal tasks. While prior research has attempted to mitigate this by modeling intra-modal ambiguity, it often overlooks inherent annotation flaws, leading to suboptimal uncertainty allocation. To address these challenges, we propose a Variational Adapter for Cross-modal Similarity Representation (VACSR). This approach reformulates image-text matching with fine-grained semantic scarcity as a variational inference problem. It constructs a latent space for cross-modal similarity and uses regularization techniques to mitigate overfitting to binary annotations. Experiments on image-text retrieval, domain generalization, and base-to-novel generalization demonstrate the proposed method's effectiveness and robust generalization ability.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。