arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1207.5649stat.ME
收藏

Netflix挑战的统计意义

Statistical Significance of the Netflix Challenge

Andrey Feuerverger, Yu He, Shashi Khatri

摘要:

受Netflix竞赛遗产的启发,我们概述了我们从自己的努力和其他人的努力中所学到的关于协同过滤和推荐系统问题的知识。该数据集包含约1亿部电影评级(从1星到5星),涉及约48万用户和约1.8万部电影;相关评级矩阵约为99%稀疏。目标是预测用户对电影的评分;能够准确做到这一点的系统具有重要的商业应用,特别是在万维网上。我们讨论,在一些细节,方法“基线”建模,奇异值分解(SVD),以及kNN(最近邻)和神经网络模型;时间效应,交叉验证问题,集成方法和其他考虑进行了讨论。我们比较现有的模型在寻找新的模型,并讨论了惩罚和参数收缩时出现的参数空间的尺寸达到数百万的关键任务的问题。尽管计算机科学和机器学习社区已经对这些问题做了很多工作,但我们的目标是解决统计受众,并提供从这组非凡的数据中吸取的教训的主要统计处理。

英文摘要:

Inspired by the legacy of the Netflix contest, we provide an overview of what has been learned---from our own efforts, and those of others---concerning the problems of collaborative filtering and recommender systems. The data set consists of about 100 million movie ratings (from 1 to 5 stars) involving some 480 thousand users and some 18 thousand movies; the associated ratings matrix is about 99% sparse. The goal is to predict ratings that users will give to movies; systems which can do this accurately have significant commercial applications, particularly on the world wide web. We discuss, in some detail, approaches to "baseline" modeling, singular value decomposition (SVD), as well as kNN (nearest neighbor) and neural network models; temporal effects, cross-validation issues, ensemble methods and other considerations are discussed as well. We compare existing models in a search for new models, and also discuss the mission-critical issues of penalization and parameter shrinkage which arise when the dimensions of a parameter space reaches into the millions. Although much work on such problems has been carried out by the computer science and machine learning communities, our goal here is to address a statistical audience, and to provide a primarily statistical treatment of the lessons that have been learned from this remarkable set of data.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑