arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2210.04149cs.IR
收藏

从评估视角重新审视推荐系统

Take a Fresh Look at Recommender Systems from an Evaluation Standpoint

Aixin Sun

AI总结:

本文从评估角度重新审视推荐系统,指出常用数据划分会导致流行度基线定义不清,并分析忽视全局时间线引发的数据泄漏与偏好建模简化问题,进而提出更贴近真实场景的评估与建模思路。

摘要:

推荐已成为信息检索(IR)领域的一个突出研究方向。评估也是该社区中的传统研究主题。受近期研究报告的若干反直觉观察所启发,这篇观点论文从评估立场出发,重新审视推荐系统。本文并非考察 recall、hit rate 或 NDCG 等指标,也不是探讨新颖性和多样性等视角,而是重点关注在评估推荐算法时这些指标是如何计算的。具体而言,论文重新审视了常用的训练/测试数据划分及其后果。我们首先考察常见的数据划分方法,例如随机划分或留一法,并讨论为什么在这些划分下流行度基线定义不清。随后,本文探讨在评估中忽视全局时间线所产生的两个影响:数据泄漏,以及用户偏好建模的过度简化。之后,我们提出关于推荐系统的新视角,包括能更准确反映真实世界场景的算法性能评估技术,以及在用户偏好建模中考虑决策情境的可能方法。

英文摘要:

Recommendation has become a prominent area of research in the field of Information Retrieval (IR). Evaluation is also a traditional research topic in this community. Motivated by a few counter-intuitive observations reported in recent studies, this perspectives paper takes a fresh look at recommender systems from an evaluation standpoint. Rather than examining metrics like recall, hit rate, or NDCG, or perspectives like novelty and diversity, the key focus here is on how these metrics are calculated when evaluating a recommender algorithm. Specifically, the commonly used train/test data splits and their consequences are re-examined. We begin by examining common data splitting methods, such as random split or leave-one-out, and discuss why the popularity baseline is poorly defined under such splits. We then move on to explore the two implications of neglecting a global timeline during evaluation: data leakage and oversimplification of user preference modeling. Afterwards, we present new perspectives on recommender systems, including techniques for evaluating algorithm performance that more accurately reflect real-world scenarios, and possible approaches to consider decision contexts in user preference modeling.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑