作者元数据与接受度之间的关联:2017-2022年期间ICLR提交语料库的功能丰富,匹配观察研究
Association between author metadata and acceptance: A feature-rich, matched observational study of a corpus of ICLR submissions between 2017-2022
摘要:
近年来,许多研究探讨了学术期刊和学术会议同行评议过程中的地位偏见。在这篇文章中,我们调查了作者元数据和地区主席的最终决定(接受/拒绝)之间的关联,使用我们汇编的数据库,其中包含2017年至2022年国际学习表征会议(ICLR)的5,313份边界提交。我们仔细定义了因果分析中的元素,包括治疗及其时间、治疗前变量、潜在结局和感兴趣的因果原假设,所有这些都是在研究单元为文本数据的背景下,并在Neyman和Rubin的潜在结局(PO)框架下进行的。我们发现一些微弱的证据表明作者元数据与文章的最终决定有关。我们还发现,在额外的稳定性假设下,来自高排名机构(前30%或前20%)的边缘文章与其匹配的同行相比,不太受地区主席的青睐。两种不同匹配设计的结果一致(第一种设计的比值比= 0.82 [95% CI:0.67 - 1.00],强化设计的比值比= 0.83 [95% CI:0.64 - 1.07])。我们讨论了如何解释这些结果的背景下,一个研究单位和不同的代理人(评审员和地区主席)在同行评审系统中的多个相互作用。
英文摘要:
Many recent studies have probed status bias in the peer-review process of academic journals and conferences. In this article, we investigated the association between author metadata and area chairs' final decisions (Accept/Reject) using our compiled database of 5,313 borderline submissions to the International Conference on Learning Representations (ICLR) from 2017 to 2022. We carefully defined elements in a cause-and-effect analysis, including the treatment and its timing, pre-treatment variables, potential outcomes and causal null hypothesis of interest, all in the context of study units being textual data and under Neyman and Rubin's potential outcomes (PO) framework. We found some weak evidence that author metadata was associated with articles' final decisions. We also found that, under an additional stability assumption, borderline articles from high-ranking institutions (top-30% or top-20%) were less favored by area chairs compared to their matched counterparts. The results were consistent in two different matched designs (odds ratio = 0.82 [95% CI: 0.67 to 1.00] in a first design and 0.83 [95% CI: 0.64 to 1.07] in a strengthened design). We discussed how to interpret these results in the context of multiple interactions between a study unit and different agents (reviewers and area chairs) in the peer-review system.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。