硬否定或假否定：纠正训练神经排名模型中的池化偏差
Hard Negatives or False Negatives: Correcting Pooling Bias in Training Neural Ranking Models
摘要：
神经排序模型已成为信息检索中最重要的技术之一。由于相关标签的限制，NRM的训练严重依赖于对未标记数据的负采样。在一般的机器学习场景中，已经表明使用硬否定（即，接近阳性的样本）可以导致更好的性能。令人惊讶的是，我们发现相反的结果，从我们的经验研究在IR。当采样排名靠前的结果（不包括标记的阳性）作为负面的一个更强大的检索，学习NRM的性能变得更糟。根据我们的调查，表面原因是有更多的假阴性（即，无标记的阳性），这可能会损害训练过程;其根源是数据集构建过程中存在池化偏差，注释器只对一些基本检索器选择的极少数样本进行判断和标记。因此，原则上，我们可以将训练NRM中的假阴性问题表述为从具有池化偏差的标记数据集学习。为了解决这个问题，我们提出了一种新的耦合估计技术（CET），同时学习相关性模型和选择模型，以纠正训练NRM的池化偏差。三个检索基准的实证结果表明，与其他基线策略相比，我们的技术训练的NRM可以实现显着的收益。
英文摘要：
Neural ranking models (NRMs) have become one of the most important techniques in information retrieval (IR). Due to the limitation of relevance labels, the training of NRMs heavily relies on negative sampling over unlabeled data. In general machine learning scenarios, it has shown that training with hard negatives (i.e., samples that are close to positives) could lead to better performance. Surprisingly, we find opposite results from our empirical studies in IR. When sampling top-ranked results (excluding the labeled positives) as negatives from a stronger retriever, the performance of the learned NRM becomes even worse. Based on our investigation, the superficial reason is that there are more false negatives (i.e., unlabeled positives) in the top-ranked results with a stronger retriever, which may hurt the training process; The root is the existence of pooling bias in the dataset constructing process, where annotators only judge and label very few samples selected by some basic retrievers. Therefore, in principle, we can formulate the false negative issue in training NRMs as learning from labeled datasets with pooling bias. To solve this problem, we propose a novel Coupled Estimation Technique (CET) that learns both a relevance model and a selection model simultaneously to correct the pooling bias for training NRMs. Empirical results on three retrieval benchmarks show that NRMs trained with our technique can achieve significant gains on ranking effectiveness against other baseline strategies.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
