arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2304.02434cs.IR
收藏

兼顾效率与效果!搜索系统中大规模预排序框架

Both Efficiency and Effectiveness! A Large Scale Pre-ranking Framework in Search System

Qihang Zhao, Rui-jie Zhu, Liu Yang, He Yongming, Bo Zhou, Luo Cheng

AI总结:

针对搜索预排序阶段表征模型缺乏交互导致效果下降的问题,提出RankDFM框架,以DeepFM为骨干并采用成对训练,通过特征交叉提升效果,并引入可学习特征选择降低推理耗时,已部署于短视频App服务数亿用户。

摘要:

在搜索系统领域,多级级联架构是一种普遍采用的方法,通常由匹配、预排序和排序等顺序模块组成。人们普遍认为,预排序阶段使用的模型必须在效果与效率之间取得平衡。因此,最常用的架构是基于表征的向量积模型。然而,这种架构缺乏查询与文档之间的有效交互,导致搜索系统效果下降。为解决这一问题,我们提出了一种名为RankDFM的新型预排序框架。该框架以DeepFM作为骨干模型,并采用成对训练范式来学习查询下视频的排序。RankDFM具备的特征交叉能力在离线评估和在线A/B测试中带来了显著的性能提升。此外,我们引入了一种可学习的特征选择方案来优化模型并减少在线推理所需的时间,其推理耗时与树模型相当。目前,RankDFM已部署于一款短视频App的搜索系统中,每天为数亿用户提供服务。

英文摘要:

In the realm of search systems, multi-stage cascade architecture is a prevalent method, typically consisting of sequential modules such as matching, pre-ranking, and ranking. It is generally acknowledged that the model used in the pre-ranking stage must strike a balance between efficacy and efficiency. Thus, the most commonly employed architecture is the representation-focused vector product based model. However, this architecture lacks effective interaction between the query and document, resulting in a reduction in the effectiveness of the search system. To address this issue, we present a novel pre-ranking framework called RankDFM. Our framework leverages DeepFM as the backbone and employs a pairwise training paradigm to learn the ranking of videos under a query. The capability of RankDFM to cross features provides significant improvement in offline and online A/B testing performance. Furthermore, we introduce a learnable feature selection scheme to optimize the model and reduce the time required for online inference, equivalent to a tree model. Currently, RankDFM has been deployed in the search system of a shortvideo App, providing daily services to hundreds of millions users.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑