AI 中文总结
该综述分析了2020至2025年ACM RecSys会议可复现性 track 的51篇论文，揭示其范围扩展、方法学特征及实践中可复现性的现状，指出可复现性概念与落地存在差距。
AI 中文摘要
可复现性已成为可信推荐系统研究的基石，这源于人们对实验结果可靠性和可推广性的日益担忧。为此，ACM RecSys会议于2020年推出了专门的可复现性 track，以鼓励严谨、透明且可重复的研究。本文对2020至2025年该 track 的51篇已接收论文进行了结构化分析，按类型对贡献进行分类，并分析了数据集、算法、框架和评估实践中的常见模式，旨在了解社区内可复现性在实践中如何落地。研究发现三大趋势：其一，该 track 范围有所扩展，从聚焦复现与复制延伸至包含基准测试、资源及方法学贡献；其二，可复现性论文呈现一致的方法学特征，依赖有限的数据集、算法和评估协议；其三，实践中的可复现性常涉及对先前实验的扩展，而非严格复制，研究常引入额外模型或评估标准。总体而言，可复现性工作提升了透明度与文档质量，但对方法学多样性的影响有限，凸显了可复现性的概念定义与其实际落地之间存在差距。
英文摘要
Reproducibility has become a cornerstone of credible recommender systems research, driven by growing concerns about the reliability and generalizability of experimental results. In response, the ACM RecSys conference introduced a dedicated Reproducibility Track in 2020 to encourage rigorous, transparent, and repeatable research. This paper presents a structured analysis of the track from 2020 to 2025, covering 51 accepted papers. We classify contributions by type and analyze common patterns in datasets, algorithms, frameworks, and evaluation practices, with the goal of understanding how reproducibility is operationalized in practice within the community. Our findings reveal three main trends. First, the track has expanded in scope, evolving from a focus on reproduction and replication to include benchmarking, resources, and methodological contributions. Second, reproducibility papers exhibit a consistent methodological profile, relying on a limited set of datasets, algorithms, and evaluation protocols. Third, reproducibility in practice often involves extending prior experiments rather than strictly replicating them, with studies frequently introducing additional models or evaluation criteria. Overall, reproducibility work has improved transparency and documentation, but has had limited impact on methodological diversity, highlighting a gap between the conceptual definition of reproducibility and its implementation.
CommentsAccepted at ACM Transactions on Recommender Systems
