arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2011.12919cs.LGcs.AIstat.ML
收藏

分析机器学习会议评审过程

Analyzing the Machine Learning Conference Review Process

University of California, Berkeley(加州大学伯克利分校);Princeton University(普林斯顿大学);Centennial High School(百年高中);Harvard University(哈佛大学);University of Maryland, College Park(马里兰大学帕克分校)

David Tran, Alex Valtchanov, Keshav Ganapathy, Raymond Feng, Eric Slud, Micah Goldblum, Tom Goldstein

摘要:

近年来,主流机器学习会议的参与者数量急剧增加,观点也越来越广泛。机器学习社区的成员可能会无意中听到从接受决定的随机性到制度偏见的指控。在这项工作中,我们通过对2017年至2020年提交给ICLR的论文进行全面研究,批判性地分析了审查过程。我们量化的再现性/随机性的审查分数和验收决定,并检查分数是否与论文的影响。我们的研究结果表明,在接受/拒绝的决定,即使在控制纸张质量强的机构偏见。此外,我们发现了性别差距的证据,女性作者的得分较低,接受率较低,每篇论文的引用次数少于男性作者。最后,我们对未来的会议组织者提出了建议。

英文摘要:

Mainstream machine learning conferences have seen a dramatic increase in the number of participants, along with a growing range of perspectives, in recent years. Members of the machine learning community are likely to overhear allegations ranging from randomness of acceptance decisions to institutional bias. In this work, we critically analyze the review process through a comprehensive study of papers submitted to ICLR between 2017 and 2020. We quantify reproducibility/randomness in review scores and acceptance decisions, and examine whether scores correlate with paper impact. Our findings suggest strong institutional bias in accept/reject decisions, even after controlling for paper quality. Furthermore, we find evidence for a gender gap, with female authors receiving lower scores, lower acceptance rates, and fewer citations per paper than their male counterparts. We conclude our work with recommendations for future conference organizers.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑