arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2405.04844cs.IR
收藏

全阶段学习排序:面向多阶段系统的统一框架

Full Stage Learning to Rank: A Unified Framework for Multi-Stage Systems

Kai Zheng, Haijun Zhao, Rui Huang, Beichuan Zhang, Na Mou, Yanan Niu, Yang Song, Hongning Wang, Kun Gai

AI总结:

本文提出 GPRP 与 Full Stage Learning to Rank 统一框架,通过估计并适配下游阶段选择偏差优化多阶段推荐排序,并在仿真和短视频平台在线 A/B 测试中验证其显著收益。

摘要:

概率排序原则(Probability Ranking Principle,PRP)一直被视为信息检索(IR)系统设计中的基础性标准。该原则要求 IR 模块返回的结果列表按照潜在的用户兴趣进行排序,从而最大化结果的效用。\n然而,我们指出,在当代 IR 系统的每个阶段中不加区分地应用 PRP 是不合适的。此类系统包含多个阶段(例如本文考察的召回、预排序、排序和重排序阶段)。每个阶段模型中固有的选择偏差会显著影响最终展示给用户的结果。\n为解决这一问题,我们针对多阶段系统提出了一种改进的排序原则,即广义概率排序原则(Generalized Probability Ranking Principle,GPRP),以同时强调系统流水线中每个阶段的选择偏差以及用户的潜在兴趣。\n我们通过一个名为 Full Stage Learning to Rank 的统一算法框架实现 GPRP。我们的核心思想是,首先估计后续阶段中的选择偏差,然后学习一个最符合下游模块选择偏差的排序模型,从而将其排名最高的结果送入系统输出的最终排序列表。\n我们使用仿真实验和在一家领先短视频推荐平台上开展的在线 A/B 测试,对所开发的 Full Stage Learning to Rank 解决方案进行了广泛实验评估。实验证明,该算法在召回和排序阶段均有效。自部署以来,该算法已为平台带来持续且显著的性能提升。

英文摘要:

The Probability Ranking Principle (PRP) has been considered as the foundational standard in the design of information retrieval (IR) systems. The principle requires an IR module's returned list of results to be ranked with respect to the underlying user interests, so as to maximize the results' utility. Nevertheless, we point out that it is inappropriate to indiscriminately apply PRP through every stage of a contemporary IR system. Such systems contain multiple stages (e.g., retrieval, pre-ranking, ranking, and re-ranking stages, as examined in this paper). The \emph{selection bias} inherent in the model of each stage significantly influences the results that are ultimately presented to users. To address this issue, we propose an improved ranking principle for multi-stage systems, namely the Generalized Probability Ranking Principle (GPRP), to emphasize both the selection bias in each stage of the system pipeline as well as the underlying interest of users. We realize GPRP via a unified algorithmic framework named Full Stage Learning to Rank. Our core idea is to first estimate the selection bias in the subsequent stages and then learn a ranking model that best complies with the downstream modules' selection bias so as to deliver its top ranked results to the final ranked list in the system's output. We performed extensive experiment evaluations of our developed Full Stage Learning to Rank solution, using both simulations and online A/B tests in one of the leading short-video recommendation platforms. The algorithm is proved to be effective in both retrieval and ranking stages. Since deployed, the algorithm has brought consistent and significant performance gain to the platform.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑