arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2003.11941cs.LGcs.AIstat.ML
收藏

AliExpress学习排序:无需在线即可最大化在线模型性能

AliExpress Learning-To-Rank: Maximizing Online Model Performance without Going Online

Alibaba(阿里巴巴);Nanjing University(南京大学);Nanyang Technological University(南洋理工大学)

Guangda Huzhang, Zhen-Jia Pang, Yongqing Gao, Yawen Liu, Weijie Shen, Wen-Ji Zhou, Qing Da, An-Xiang Zeng, Han Yu, Yang Yu, Zhi-Hua Zhou

摘要:

学习排序(LTR)已成为电子商务应用中的关键技术。大多数现有的LTR方法遵循从在线系统收集的离线标记数据的监督学习范例。然而,人们已经注意到,以前的LTR模型可以在离线验证数据上具有良好的验证性能,但在线性能较差,反之亦然,这意味着离线和在线评估之间可能存在很大的不一致性。我们在本文中调查并确认,这种不一致性存在,并可能对全球速卖通搜索产生重大影响。不一致的原因包括在学习过程中忽略项目上下文,离线数据集不足以学习上下文。因此,本文提出了一个评估器生成器框架LTR与项目上下文。该框架包括一个评估者,概括评估涉及的背景下,和一个生成器,最大限度地提高评估者的分数,通过强化学习,和一个神经网络,确保评估者的泛化。在模拟环境和全球速卖通搜索在线系统中进行的大量实验表明,首先,离线数据集上的经典数据基指标可能与在线性能存在显著不一致,甚至可能产生误导。其次,建议的评估者分数是显着更一致的在线性能比常见的排名指标。最后,作为结果,我们的方法实现了显着的改善(\textgreater $2\%$)的转换率(CR)在工业级微调模型在线A/B测试。

英文摘要:

Learning-to-rank (LTR) has become a key technology in E-commerce applications. Most existing LTR approaches follow a supervised learning paradigm from offline labeled data collected from the online system. However, it has been noticed that previous LTR models can have a good validation performance over offline validation data but have a poor online performance, and vice versa, which implies a possible large inconsistency between the offline and online evaluation. We investigate and confirm in this paper that such inconsistency exists and can have a significant impact on AliExpress Search. Reasons for the inconsistency include the ignorance of item context during the learning, and the offline data set is insufficient for learning the context. Therefore, this paper proposes an evaluator-generator framework for LTR with item context. The framework consists of an evaluator that generalizes to evaluate recommendations involving the context, and a generator that maximizes the evaluator score by reinforcement learning, and a discriminator that ensures the generalization of the evaluator. Extensive experiments in simulation environments and AliExpress Search online system show that, firstly, the classic data-based metrics on the offline dataset can show significant inconsistency with online performance, and can even be misleading. Secondly, the proposed evaluator score is significantly more consistent with the online performance than common ranking metrics. Finally, as the consequence, our method achieves a significant improvement (\textgreater$2\%$) in terms of Conversion Rate (CR) over the industrial-level fine-tuned model in online A/B tests.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑