arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2101.04526cs.LGcs.CYcs.IR
收藏

用模拟用户测量推荐系统效果

Measuring Recommender System Effects with Simulated Users

Virginia Tech(弗吉尼亚理工大学);Google(谷歌)

Sirui Yao, Yoni Halpern, Nithum Thain, Xuezhi Wang, Kang Lee, Flavien Prost, Ed H. Chi, Jilin Chen, Alex Beutel

摘要:

想象一下一个食物推荐系统--我们如何检查它是\emph{causing},培养不健康的饮食习惯还是仅仅反映用户的兴趣?随着时间的推移,用户对推荐系统的体验有多少是由推荐系统的选择和偏见引起的,有多少是基于用户的偏好和偏见?流行度偏差和过滤气泡是两个最好的研究推荐系统的偏见,但大多数以前的研究集中在理解系统的行为在一个单一的推荐步骤。这些偏见是如何与用户行为相互作用的,重复的交互会产生什么类型的用户体验?在这项工作中,我们提供了一个模拟的框架来衡量不同类型的用户行为下的推荐系统的影响。使用这个模拟框架,我们可以(a)从用户偏好中分离推荐系统的效果,以及(b)检查系统如何表现不仅是平均的“平均用户”,而且在非典型用户行为下的极端体验。作为模拟框架的一部分,我们提出了一套评估指标的模拟,以了解推荐系统的行为。最后,我们提出了两个实证案例研究-一个在MovieLens传统的协同过滤和一个大规模的生产推荐系统-了解如何随着时间的推移流行的偏见体现。

英文摘要:

Imagine a food recommender system -- how would we check if it is \emph{causing} and fostering unhealthy eating habits or merely reflecting users' interests? How much of a user's experience over time with a recommender is caused by the recommender system's choices and biases, and how much is based on the user's preferences and biases? Popularity bias and filter bubbles are two of the most well-studied recommender system biases, but most of the prior research has focused on understanding the system behavior in a single recommendation step. How do these biases interplay with user behavior, and what types of user experiences are created from repeated interactions? In this work, we offer a simulation framework for measuring the impact of a recommender system under different types of user behavior. Using this simulation framework, we can (a) isolate the effect of the recommender system from the user preferences, and (b) examine how the system performs not just on average for an "average user" but also the extreme experiences under atypical user behavior. As part of the simulation framework, we propose a set of evaluation metrics over the simulations to understand the recommender system's behavior. Finally, we present two empirical case studies -- one on traditional collaborative filtering in MovieLens and one on a large-scale production recommender system -- to understand how popularity bias manifests over time.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑