arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2309.05892cs.IRcs.HC
收藏

基于分布信息的推荐系统评估

Distributionally-Informed Recommender System Evaluation

Michael D. Ekstrand, Ben Carterette, Fernando Diaz

AI总结:

本文主张在推荐系统评估中需关注效用分布与不确定性来源,强调利用分布思维设计评估过程,以确保系统公平分配收益。

摘要:

当前评估推荐系统的做法通常集中于用户导向的有效性指标或商业指标的点估计,有时结合多样性和新颖性等额外指标进行考量。在本文中,我们主张研究人员和从业者需要更加密切关注推荐系统(或其他信息访问系统)产生的各种分布,以及导致这些分布的不确定性来源。我们论点的一个直接含义是,研究人员和从业者都必须更彻底地报告和检查不同利益相关者群体之间及内部的效用分布。然而,各种形式的分布在推荐系统实验过程的更多方面出现,分布思维对我们如何设计、评估和呈现推荐系统评估及研究结果具有重大影响。在推荐系统评估中利用和强调分布是确保系统向受其影响的人们提供适当且公平分配的收益的必要步骤。

英文摘要:

Current practice for evaluating recommender systems typically focuses on point estimates of user-oriented effectiveness metrics or business metrics, sometimes combined with additional metrics for considerations such as diversity and novelty. In this paper, we argue for the need for researchers and practitioners to attend more closely to various distributions that arise from a recommender system (or other information access system) and the sources of uncertainty that lead to these distributions. One immediate implication of our argument is that both researchers and practitioners must report and examine more thoroughly the distribution of utility between and within different stakeholder groups. However, distributions of various forms arise in many more aspects of the recommender systems experimental process, and distributional thinking has substantial ramifications for how we design, evaluate, and present recommender systems evaluation and research results. Leveraging and emphasizing distributions in the evaluation of recommender systems is a necessary step to ensure that the systems provide appropriate and equitably-distributed benefit to the people they affect.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑