arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2205.09394cs.IR
收藏

AutoFAS:预排序系统的自动特征和架构选择

AutoFAS: Automatic Feature and Architecture Selection for Pre-Ranking System

Xiang Li, Xiaojiang Zhou, Yao Xiao, Peihao Huang, Dayao Chen, Sheng Chen, Yunsen Xian

摘要:

工业搜索和推荐系统大多遵循经典的多阶段信息检索范式:匹配、预排名、排名和重新排名阶段。为了考虑系统效率,通常在预排序阶段部署基于简单向量乘积的模型。最近的工作考虑蒸馏大排名模型的高知识,以更好的效果小预排名模型。然而,在预排名系统中仍然存在两个主要的挑战:(i)在没有显式地对性能增益与计算成本进行建模的情况下,预排名阶段中的预定义延迟约束不可避免地导致次优解决方案;(ii)将排名教师的知识传递给具有预定手工架构的预排名学生仍然遭受模型性能的损失。本文提出了一种新的AutoFAS框架,该框架能够联合优化预排序模型的效率和有效性:(i)AutoFAS首次使用神经结构搜索(NAS)技术同时选择最有价值的特征和网络结构;(ii)在NAS过程期间配备有排名模型引导的奖励,AutoFAS可以为给定的排名教师选择最佳的预排名架构,而无需任何计算开销。在我们的真实世界的搜索系统中的实验结果表明,AutoFAS始终优于以前的国家的最先进的(SOTA)的方法,以较低的计算成本。值得注意的是,我们的模型已被应用于美团搜索系统的预排名模块,带来了显著的改进。

英文摘要:

Industrial search and recommendation systems mostly follow the classic multi-stage information retrieval paradigm: matching, pre-ranking, ranking, and re-ranking stages. To account for system efficiency, simple vector-product based models are commonly deployed in the pre-ranking stage. Recent works consider distilling the high knowledge of large ranking models to small pre-ranking models for better effectiveness. However, two major challenges in pre-ranking system still exist: (i) without explicitly modeling the performance gain versus computation cost, the predefined latency constraint in the pre-ranking stage inevitably leads to suboptimal solutions; (ii) transferring the ranking teacher's knowledge to a pre-ranking student with a predetermined handcrafted architecture still suffers from the loss of model performance. In this work, a novel framework AutoFAS is proposed which jointly optimizes the efficiency and effectiveness of the pre-ranking model: (i) AutoFAS for the first time simultaneously selects the most valuable features and network architectures using Neural Architecture Search (NAS) technique; (ii) equipped with ranking model guided reward during NAS procedure, AutoFAS can select the best pre-ranking architecture for a given ranking teacher without any computation overhead. Experimental results in our real world search system show AutoFAS consistently outperforms the previous state-of-the-art (SOTA) approaches at a lower computing cost. Notably, our model has been adopted in the pre-ranking module in the search system of Meituan, bringing significant improvements.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑