开源复制的警笛之歌
A Siren Song of Open Source Reproducibility
摘要:
随着可重复性变得越来越受关注,会议在很大程度上集中于要求评审员指出代码是否附加到提交中的策略。这是一个更大的趋势的一部分,即根据假设的理想采取行动,而不研究这些行动是否会产生预期的结果。我们的论点是,如果我们想改善可复制研究的状况,那么这种对复制代码的关注是错误的。这种关注可能是有害的--我们不应该强制提交代码。没有证据表明会议采取了有效行动来鼓励和奖励重复性。我们认为,场馆必须采取更多的行动,以推进可重复的机器学习研究今天。
英文摘要:
As reproducibility becomes a greater concern, conferences have largely converged to a strategy of asking reviewers to indicate whether code was attached to a submission. This is part of a larger trend of taking action based on assumed ideals, without studying if those actions will yield the desired outcome. Our argument is that this focus on code for replication is misguided if we want to improve the state of reproducible research. This focus can be harmful -- we should not force code to be submitted. There is a lack of evidence for effective actions taken by conferences to encourage and reward reproducibility. We argue that venues must take more action to advance reproducible machine learning research today.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。