破解双盲评审:基于深度学习的作者身份归属
Cracking Double-Blind Review: Authorship Attribution with Deep Learning
AI总结:
提出基于transformer的神经网络架构,利用文本和参考文献作者姓名进行匿名手稿作者归属,在arXiv超200万篇论文数据集上达到73%准确率,揭示了双盲评审的潜在风险。
摘要:
双盲同行评审被认为是学术研究的支柱,因为人们认为它能确保公平、无偏见且以事实为中心的科学讨论。然而,经验丰富的研究人员通常能准确猜出匿名提交的论文出自哪个研究组,从而使同行评审过程产生偏见。在这项工作中,我们提出了一种基于transformer的神经网络架构,仅使用文本内容和参考文献中的作者姓名,将匿名手稿归属于特定作者。为了训练和评估我们的方法,我们创建了迄今为止最大的作者身份识别数据集。它利用了arXiv上公开提供的所有研究论文,总计超过200万篇手稿。在包含多达2,000名不同作者的arXiv子集中,我们的方法实现了前所未有的作者身份归属准确率,高达73%的论文被正确归属。我们提出了扩展性分析,以强调当充足的计算能力更广泛地提供给学术界时,所提方法对更大数据集的适用性。此外,我们还在旨在识别匿名手稿所有作者的场景中分析了归属准确率。得益于我们的方法,我们不仅能预测匿名作品的作者,还提供了使论文可被归属的关键方面的实证证据。我们已经开源了重现实验所需的工具。
英文摘要:
Double-blind peer review is considered a pillar of academic research because it is perceived to ensure a fair, unbiased, and fact-centered scientific discussion. Yet, experienced researchers can often correctly guess from which research group an anonymous submission originates, biasing the peer-review process. In this work, we present a transformer-based, neural-network architecture that only uses the text content and the author names in the bibliography to attribute an anonymous manuscript to an author. To train and evaluate our method, we created the largest authorship identification dataset to date. It leverages all research papers publicly available on arXiv amounting to over 2 million manuscripts. In arXiv-subsets with up to 2,000 different authors, our method achieves an unprecedented authorship attribution accuracy, where up to 73% of papers are attributed correctly. We present a scaling analysis to highlight the applicability of the proposed method to even larger datasets when sufficient compute capabilities are more widely available to the academic community. Furthermore, we analyze the attribution accuracy in settings where the goal is to identify all authors of an anonymous manuscript. Thanks to our method, we are not only able to predict the author of an anonymous work, but we also provide empirical evidence of the key aspects that make a paper attributable. We have open-sourced the necessary tools to reproduce our experiments.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。