ReCon：通过关系一致性增强真实对应判别以实现鲁棒的噪声对应学习
ReCon: Enhancing True Correspondence Discrimination through Relation Consistency for Robust Noisy Correspondence Learning
AI总结：
本文提出关系一致性学习框架ReCon，通过跨模态与模态内双重关系一致性约束，提升多模态数据中真实对应判别能力，从而在噪声对应学习场景下有效过滤不匹配数据对，并在三个基准数据集上验证了其优越性。
摘要：
我们能否从包含不匹配数据对的多模态数据集中准确识别真实对应关系？现有方法主要强调跨模态对象表示之间的相似性匹配，可能忽略了模态内部的关系一致性，而这种一致性对于区分真假对应关系尤为重要。这一疏漏往往会导致将负样本误判为正样本的风险，从而引发意想不到的性能下降。为解决该问题，我们提出了一个通用的关系一致性学习框架，即ReCon，以准确判别多模态数据中的真实对应关系，从而有效缓解不匹配所带来的不利影响。具体而言，ReCon利用一种新颖的关系一致性学习来确保双重对齐，分别是不同模态之间的跨模态关系一致性以及模态内部的模态内关系一致性。得益于这种对关系的双重约束，ReCon显著增强了其在真实对应判别方面的有效性，因而能够可靠地过滤掉不匹配数据对，以降低错误监督的风险。我们在三个广泛使用的基准数据集上进行了大量实验，包括Flickr30K、MS-COCO和Conceptual Captions，证明了ReCon相较于其他SOTA方法的有效性和优越性。代码可在以下地址获取：https://github.com/qxzha/ReCon。
英文摘要：
Can we accurately identify the true correspondences from multimodal datasets containing mismatched data pairs? Existing methods primarily emphasize the similarity matching between the representations of objects across modalities, potentially neglecting the crucial relation consistency within modalities that are particularly important for distinguishing the true and false correspondences. Such an omission often runs the risk of misidentifying negatives as positives, thus leading to unanticipated performance degradation. To address this problem, we propose a general Relation Consistency learning framework, namely ReCon, to accurately discriminate the true correspondences among the multimodal data and thus effectively mitigate the adverse impact caused by mismatches. Specifically, ReCon leverages a novel relation consistency learning to ensure the dual-alignment, respectively of, the cross-modal relation consistency between different modalities and the intra-modal relation consistency within modalities. Thanks to such dual constrains on relations, ReCon significantly enhances its effectiveness for true correspondence discrimination and therefore reliably filters out the mismatched pairs to mitigate the risks of wrong supervisions. Extensive experiments on three widely-used benchmark datasets, including Flickr30K, MS-COCO, and Conceptual Captions, are conducted to demonstrate the effectiveness and superiority of ReCon compared with other SOTAs. The code is available at: https://github.com/qxzha/ReCon.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
