我们真的取得了很大的进步吗?最近的神经推荐方法令人担忧的分析
Are We Really Making Much Progress? A Worrying Analysis of Recent Neural Recommendation Approaches
摘要:
深度学习技术已经成为研究推荐系统算法方面的研究人员的首选方法。随着一般对机器学习的兴趣的强烈增加,因此,很难跟踪目前代表最先进技术的东西,例如,用于top-n推荐任务。与此同时,最近的一些出版物指出了当今应用机器学习研究实践中的问题,例如,在提出新模型时,在结果的可重复性或基线的选择方面。在这项工作中,我们报告了一个系统的分析结果的算法建议前n推荐任务。具体来说,我们考虑了过去几年在顶级研究会议上提出的18种算法。其中只有7个可以通过合理的努力复制。然而,对于这些方法,事实证明,其中6种方法通常可以优于非常简单的探试法,例如,基于最近邻或基于图的技术。剩下的一个明显优于基线,但并没有一贯优于一个良好的调整非神经线性排名方法。总的来说,我们的工作揭示了当今机器学习奖学金中的一些潜在问题,并呼吁改进这一领域的科学实践。我们实验的源代码和完整结果可在https://github.com/MaurizioFD/RecSys2019_DeepLearning_Evaluation上获得。
英文摘要:
Deep learning techniques have become the method of choice for researchers working on algorithmic aspects of recommender systems. With the strongly increased interest in machine learning in general, it has, as a result, become difficult to keep track of what represents the state-of-the-art at the moment, e.g., for top-n recommendation tasks. At the same time, several recent publications point out problems in today's research practice in applied machine learning, e.g., in terms of the reproducibility of the results or the choice of the baselines when proposing new models. In this work, we report the results of a systematic analysis of algorithmic proposals for top-n recommendation tasks. Specifically, we considered 18 algorithms that were presented at top-level research conferences in the last years. Only 7 of them could be reproduced with reasonable effort. For these methods, it however turned out that 6 of them can often be outperformed with comparably simple heuristic methods, e.g., based on nearest-neighbor or graph-based techniques. The remaining one clearly outperformed the baselines but did not consistently outperform a well-tuned non-neural linear ranking method. Overall, our work sheds light on a number of potential problems in today's machine learning scholarship and calls for improved scientific practices in this area. Source code of our experiments and full results are available at: https://github.com/MaurizioFD/RecSys2019_DeepLearning_Evaluation.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。