arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2204.04372cs.LGcs.AIcs.SE
收藏

开源复制的警笛之歌

A Siren Song of Open Source Reproducibility

Booz Allen Hamilton(博思艾伦咨询公司);University of Maryland, Baltimore County(马里兰大学巴尔的摩县分校)

Edward Raff, Andrew L. Farris

摘要:

随着可重复性变得越来越受关注,会议在很大程度上集中于要求评审员指出代码是否附加到提交中的策略。这是一个更大的趋势的一部分,即根据假设的理想采取行动,而不研究这些行动是否会产生预期的结果。我们的论点是,如果我们想改善可复制研究的状况,那么这种对复制代码的关注是错误的。这种关注可能是有害的--我们不应该强制提交代码。没有证据表明会议采取了有效行动来鼓励和奖励重复性。我们认为,场馆必须采取更多的行动,以推进可重复的机器学习研究今天。

英文摘要:

As reproducibility becomes a greater concern, conferences have largely converged to a strategy of asking reviewers to indicate whether code was attached to a submission. This is part of a larger trend of taking action based on assumed ideals, without studying if those actions will yield the desired outcome. Our argument is that this focus on code for replication is misguided if we want to improve the state of reproducible research. This focus can be harmful -- we should not force code to be submitted. There is a lack of evidence for effective actions taken by conferences to encourage and reward reproducibility. We argue that venues must take more action to advance reproducible machine learning research today.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑