arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1708.08447cs.IR
收藏

在评估推荐系统算法时,是时候考虑“时间”了[建议]

It's Time to Consider "Time" when Evaluating Recommender-System Algorithms [Proposal]

Joeran Beel

摘要:

在这份立场文件中,我们质疑目前将推荐系统的评估指标计算为单个数字的做法(例如,精度p= 0.28或平均绝对误差MAE = 1.21)。我们认为,单一的数字只表示平均有效性在一个通常相当长的时期(例如一年甚至更长),这只提供了一个模糊和静态的数据视图。我们建议推荐系统的研究人员应该计算时间序列的指标,如周或月,并绘制结果,例如一个折线图。通过这种方式,结果显示了算法的有效性如何随着时间的推移而发展,因此这些结果可以得出关于算法未来表现的更有意义的结论。在本文中,我们解释了我们的推理,提供了一个例子来说明我们的推理,并提出建议,社区应该做的下一步。

英文摘要:

In this position paper, we question the current practice of calculating evaluation metrics for recommender systems as single numbers (e.g. precision p=.28 or mean absolute error MAE = 1.21). We argue that single numbers express only average effectiveness over a usually rather long period (e.g. a year or even longer), which provides only a vague and static view of the data. We propose that recommender-system researchers should instead calculate metrics for time-series such as weeks or months, and plot the results in e.g. a line chart. This way, results show how algorithms' effectiveness develops over time, and hence the results allow drawing more meaningful conclusions about how an algorithm will perform in the future. In this paper, we explain our reasoning, provide an example to illustrate our reasoning and present suggestions for what the community should do next.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑