AI 中文总结
该研究针对健康科学领域同行评审的AI滥用问题，提出三重盲多LLM本地预筛选框架，保留人类最终裁决权，可减少评审延迟且更透明合规。
AI 中文摘要
学术同行评审正面临日益严峻的压力：NeurIPS 2025收到21575篇投稿，ICLR 2025收到11603篇，ICML 2025收到12107篇。这一投稿量已超出合格评审人员的供给，而大型语言模型（LLM）已在很大程度上未被披露地填补了这一缺口。对ICLR 2026的一项独立分析发现，其75800篇同行评审中约21%为完全由AI生成，超过一半显示存在一定程度的AI参与（高于2024年的15.8%）。已记录的风险包括已接收论文中出现的幻觉引用，以及手稿中嵌入的隐藏提示注入指令，以操纵AI评审人员给出有利评估。我们提出了一种为某健康科学期刊开发的三重盲多LLM同行评审预筛选框架，该框架规范并披露AI参与情况，同时保留人类评审人员作为最终决策权威。该框架将一篇投稿路由至五个阶段：去污染/匿名化、并行AI预筛选、自动检查门、盲人类评审以及编辑裁决，在检查和编辑阶段设有返回作者的循环。针对NIH/NSF禁止向第三方生成式AI提交未发表提案背后的保密担忧，所有三个AI评审人员均运行于本地托管的开源权重LLM上，将手稿内容保留在期刊基础设施内。最接近的先例Shen等人，在200篇手稿的四分位分类上对五个开源LLM进行了基准测试，发现其准确性（精确匹配率35%）不足以用于自主使用，这支持了我们保留强制人类裁决的决定。这种透明的人类监督设计为当前同行评审中不透明、不受监管的AI使用提供了一种可辩护的替代方案，有望减少传统评审的大量延迟（首次决定平均13周），同时不取代人类判断。
英文摘要
Academic peer review is under mounting strain: NeurIPS 2025 received 21,575 submissions, ICLR 2025 received 11,603, and ICML 2025 received 12,107. This volume has outpaced the supply of qualified reviewers, and large language models (LLMs) are already filling the gap, largely undisclosed. An independent analysis of ICLR 2026 found roughly 21% of its 75,800 peer reviews were fully AI-generated, with over half showing some AI involvement (up from 15.8% in 2024). Documented risks include hallucinated citations in accepted papers and hidden prompt-injection instructions embedded in manuscripts to manipulate AI reviewers into favorable assessments. We propose a triple-blind, multi-LLM pre-screening framework for peer review, developed for a health sciences journal, that formalizes and discloses AI involvement while preserving human reviewers as the final decision-making authority. The framework routes a submission through five stages -- sanitization/anonymization, parallel AI pre-screening, an automated check gate, blinded human review, and editorial adjudication -- with return-to-author loops at the check and editor stages. Addressing the confidentiality concerns behind NIH/NSF bans on submitting unpublished proposals to third-party generative AI, all three AI reviewers run on locally-hosted, open-weight LLMs, keeping manuscript content within the journal infrastructure. The closest precedent, Shen et al., benchmarked five open-source LLMs on quartile classification of 200 manuscripts and found accuracy insufficient (35% exact-match) for autonomous use, supporting our decision to retain mandatory human adjudication. This transparent, human-supervised design offers a defensible alternative to today's opaque, unregulated AI use in peer review, potentially reducing the substantial delay of traditional review (avg. 13 weeks to first decision) without displacing human judgment.
Comments16 pages, 1 figure
