多级回收管道中BERT再培训的反思
Rethink Training of BERT Rerankers in Multi-Stage Retrieval Pipeline
摘要：
预训练的深度语言模型~（LM）推进了文本检索的发展。从深度LM微调的重排序器基于丰富的上下文匹配信号来估计候选相关性。同时，深度LM也可以用来改善搜索索引，构建具有更好召回率的检索器。人们会期望两者在管道中的直接组合具有附加的性能增益。在本文中，我们发现，否则，流行的reranker不能充分利用改进的检索结果。因此，我们提出了一个局部对比估计（LCE）的训练rerankers，并证明它显着改善了深两阶段模型。
英文摘要：
Pre-trained deep language models~(LM) have advanced the state-of-the-art of text retrieval. Rerankers fine-tuned from deep LM estimates candidate relevance based on rich contextualized matching signals. Meanwhile, deep LMs can also be leveraged to improve search index, building retrievers with better recall. One would expect a straightforward combination of both in a pipeline to have additive performance gain. In this paper, we discover otherwise and that popular reranker cannot fully exploit the improved retrieval result. We, therefore, propose a Localized Contrastive Estimation (LCE) for training rerankers and demonstrate it significantly improves deep two-stage models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
