超越对比学习：合成数据实现多级相关性的列表式训练
Beyond Contrastive Learning: Synthetic Data Enables List-wise Training with Multiple Levels of Relevance
AI总结：
针对对比学习忽略文档多级相关性的问题，提出利用大语言模型生成多级相关性合成文档，并采用 Wasserstein 距离训练检索器，显著提升了 IR 性能与鲁棒性。
摘要：
尽管合成数据已经改变了信息检索（IR）流程的诸多方面，但主要的训练范式依然未变：使用二元相关性标签的对比学习，即通过 InfoNCE 损失将一个正样本文档与若干负样本文档进行比较。该目标将所有未被显式标注为相关的文档一视同仁地视为负样本，而不管其实际的相关程度如何，从而错失了对排序有用的细微差异。为了克服这一局限性，在本研究中，我们摒弃真实文档和标注，使用大语言模型直接生成合成文档，根据多个不同的相关性级别来回答 MS MARCO 查询。我们还提出使用 Wasserstein 距离作为一种更有效的损失函数，用于训练带有分级相关性标签的基于 transformer 的检索器。我们在 MS MARCO 和 BEIR 基准上的实验表明，我们提出的方法大幅优于使用 InfoNCE 的常规训练。在不使用任何真实文档的情况下，我们的方法显著提升了自监督检索器的性能，并且与使用真实数据的对比学习相比，对分布偏移具有更强的鲁棒性。我们的方法还成功地将现有的真实数据整合到合成排序上下文中，进一步提升了性能。总体而言，我们表明，生成多级排序上下文是比仅生成标准正负样本文档更好的 IR 合成数据生成方法。
英文摘要：
Although synthetic data has changed various aspects of information retrieval (IR) pipelines, the main training paradigm remains: contrastive learning with binary relevance labels, where one positive document is compared against several negatives using the InfoNCE loss. This objective treats all documents that are not explicitly annotated as relevant on an equally negative footing, regardless of their actual degree of relevance, thus missing subtle nuances useful for ranking. To overcome this limitation, in this work, we forgo real documents and annotations and use large language models to directly generate synthetic documents that answer the MS MARCO queries according to several different levels of relevance. We also propose using Wasserstein distance as a more effective loss function for training transformer-based retrievers with graduated relevance labels. Our experiments on MS MARCO and BEIR benchmark show that our proposed approach outperforms conventional training with InfoNCE by a large margin. Without using any real documents, our method significantly improves self-supervised retrievers and is more robust to distribution shift compared to contrastive learning using real data. Our method also successfully integrates existing real data into the synthetic ranking context, further boosting the performance. Overall, we show that generating multi-level ranking contexts is a better approach to synthetic data generation for IR than just generating the standard positive and negative documents.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
