Unifying Online and Counterfactual Learning to Rank
Harrie Oosterhuis, Maarten de Rijke
备注：
Harrie Oosterhuis and Maarten de Rijke. 2021. Unifying Online and Counterfactual Learning to Rank: A Novel Counterfactual Estimator that Effectively Utilizes Online Interventions. In The 14th ACM International Conference on Web Search and Data Mining (WSDM '21), March 8-12, 2021, Jerusalem, Israel. ACM, New York, NY, USA, 9 pages. https://doi.org/10.1145/3437963.3441794
Optimizing ranking systems based on user interactions is a well-studied problem. State-of-the-art methods for optimizing ranking systems based on user interactions are divided into online approaches - that learn by directly interacting with users - and counterfactual approaches - that learn from historical interactions. Existing online methods are hindered without online interventions and thus should not be applied counterfactually. Conversely, counterfactual methods cannot directly benefit from online interventions. We propose a novel intervention-aware estimator for both counterfactual and online Learning to Rank (LTR). With the introduction of the intervention-aware estimator, we aim to bridge the online/counterfactual LTR division as it is shown to be highly effective in both online and counterfactual scenarios. The estimator corrects for the effect of position bias, trust bias, and item-selection bias by using corrections based on the behavior of the logging policy and on online interventions: changes to the logging policy made during the gathering of click data. Our experimental results, conducted in a semi-synthetic experimental setup, show that, unlike existing counterfactual LTR methods, the intervention-aware estimator can greatly benefit from online interventions.
World Model-Guided Reinforcement Learning via Counterfactual User Engagement Simulation
The Chinese University of Hong Kong（香港中文大学）；MoE Key Lab of High Confidence Software Technologies, CUHK（香港中文大学 教育部高可信软件技术重点实验室）；ByteDance China（字节跳动中国）；Zhejiang University（浙江大学）
