arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2206.02593cs.LGcs.AIcs.IR
收藏

面向学习排序的悲观离线策略优化

Pessimistic Off-Policy Optimization for Learning to Rank

Brno University of Technology(布尔诺理工大学);Kempelen Institute of Intelligent Technologies(肯佩伦智能技术研究所);Amazon(亚马逊)

Matej Cief, Branislav Kveton, Michal Kompan

AI总结:

本文提出一种面向学习排序的悲观离线策略优化方法,通过点击模型参数的下置信界选择最高悲观价值列表,并结合经验贝叶斯,在推荐离线数据不平衡场景下优于基于逆倾向得分或忽略不确定性的基线方法。

摘要:

离线策略学习是一种无需部署策略即可利用另一策略收集的数据来优化策略的框架。在推荐系统中,由于记录数据存在不平衡,这一任务尤其具有挑战性:某些物品被推荐得更多,因此被记录的频率也高于其他物品。当推荐的是物品列表时,由于动作空间是组合式的,这种不平衡会进一步加剧。为应对这一挑战,我们研究了面向学习排序的悲观离线策略优化。其核心思想是计算点击模型参数的下置信界,然后返回具有最高悲观价值估计的列表。该方法计算效率高,我们对其进行了分析。我们研究了其贝叶斯变体和频率学派变体,并通过引入经验贝叶斯克服了未知先验的限制。为展示我们方法的实证效果,我们将其与使用逆倾向得分或忽略不确定性的离线优化器进行了比较。我们的方法优于所有基线,并且既稳健又具有通用性。

英文摘要:

Off-policy learning is a framework for optimizing policies without deploying them, using data collected by another policy. In recommender systems, this is especially challenging due to the imbalance in logged data: some items are recommended and thus logged more frequently than others. This is further perpetuated when recommending a list of items, as the action space is combinatorial. To address this challenge, we study pessimistic off-policy optimization for learning to rank. The key idea is to compute lower confidence bounds on parameters of click models and then return the list with the highest pessimistic estimate of its value. This approach is computationally efficient, and we analyze it. We study its Bayesian and frequentist variants and overcome the limitation of unknown prior by incorporating empirical Bayes. To show the empirical effectiveness of our approach, we compare it to off-policy optimizers that use inverse propensity scores or neglect uncertainty. Our approach outperforms all baselines and is both robust and general.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑