RankCLIP: 排序一致的语言-图像预训练
RankCLIP: Ranking-Consistent Language-Image Pretraining
AI总结：
针对CLIP等自监督对比学习模型依赖严格一对一映射、忽略图文复杂关联的问题，提出预训练方法RankCLIP，通过列表式损失及模态内、跨模态排序一致性捕捉多对多关系，在零样本分类等下游任务上取得显著提升。
摘要：
自监督对比学习模型（如CLIP）已在诸多下游任务中为视觉语言模型设立了新基准。然而，这类模型依赖严格的一对一映射，忽略了文本与图像之间及各自内部复杂且通常多面的关联。为此，我们提出RankCLIP——一种新型预训练方法，突破了CLIP及其变体的严格一对一匹配框架。通过将传统的成对损失扩展为列表式损失，并同时利用模态内与跨模态的排序一致性，RankCLIP改进了对齐过程，使其能够捕捉各模态之间及内部细微的多对多关系。通过全面实验，我们验证了RankCLIP在多种下游任务中的有效性，尤其在零样本分类任务上相较现有最优方法取得了显著提升，凸显了这一增强学习过程的重要性。
英文摘要：
Self-supervised contrastive learning models, such as CLIP, have set new benchmarks for vision-language models in many downstream tasks. However, their dependency on rigid one-to-one mappings overlooks the complex and often multifaceted relationships between and within texts and images. To this end, we introduce RankCLIP, a novel pre-training method that extends beyond the rigid one-to-one matching framework of CLIP and its variants. By extending the traditional pair-wise loss to list-wise, and leveraging both in-modal and cross-modal ranking consistency, RankCLIP improves the alignment process, enabling it to capture the nuanced many-to-many relationships between and within each modality. Through comprehensive experiments, we demonstrate the effectiveness of RankCLIP in various downstream tasks, notably achieving significant gains in zero-shot classifications over state-of-the-art methods, underscoring the importance of this enhanced learning process.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
