基于大语言模型重排序的排序列表截断
Ranked List Truncation for Large Language Model-based Re-Ranking
AI总结：
本文从“检索后重排序”视角研究排序列表截断（RLT），在TREC数据集上复现并评估了8种RLT方法在3种检索器和2种重排序器组合下的表现，为LLM重排序优化提供了新见解。
摘要：
我们从一个新颖的“检索后重排序”视角研究排序列表截断（RLT），其中我们通过截断检索到的列表（即修剪重排序候选者）来优化重排序。RLT对重排序至关重要，因为它可以通过基于每个查询向重排序器发送可变长度的候选列表来提高重排序效率。它还有可能提高重排序的有效性。尽管其很重要，但将RLT方法应用于这一新视角的研究有限。为了解决这一研究空白，我们在重排序的背景下，特别是新出现的大语言模型（LLM）重排序的背景下，复现了现有的RLT方法。具体而言，我们从三个视角考察了关于检索的RLT的既有发现在多大程度上可以推广到“检索后重排序”设置中：(i) 在具有词汇第一阶段检索的LLM重排序背景下评估RLT方法，(ii) 研究不同类型的第一阶段检索器对RLT方法的影响，以及(iii) 研究不同类型的重排序器对RLT方法的影响。我们在TREC 2019和2020深度学习赛道上进行了实验，针对涉及3个检索器和2个重排序器的流程研究了8种RLT方法。我们获得了关于重排序背景下RLT方法的新见解。
英文摘要：
We study ranked list truncation (RLT) from a novel "retrieve-then-re-rank" perspective, where we optimize re-ranking by truncating the retrieved list (i.e., trim re-ranking candidates). RLT is crucial for re-ranking as it can improve re-ranking efficiency by sending variable-length candidate lists to a re-ranker on a per-query basis. It also has the potential to improve re-ranking effectiveness. Despite its importance, there is limited research into applying RLT methods to this new perspective. To address this research gap, we reproduce existing RLT methods in the context of re-ranking, especially newly emerged large language model (LLM)-based re-ranking. In particular, we examine to what extent established findings on RLT for retrieval are generalizable to the "retrieve-then-re-rank" setup from three perspectives: (i) assessing RLT methods in the context of LLM-based re-ranking with lexical first-stage retrieval, (ii) investigating the impact of different types of first-stage retrievers on RLT methods, and (iii) investigating the impact of different types of re-rankers on RLT methods. We perform experiments on the TREC 2019 and 2020 deep learning tracks, investigating 8 RLT methods for pipelines involving 3 retrievers and 2 re-rankers. We reach new insights into RLT methods in the context of re-ranking.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
