当AI评审科学:我们能信任审稿人吗?
When AI reviews science: Can we trust the referee?
AI总结:
针对AI审稿的安全性和可靠性问题,本文通过分类攻击类型并实验验证声望框架、断言强度、反驳谄媚和上下文投毒对评分的影响,为评估AI同行评审的可靠性提供基线。
摘要:
科学投稿数量持续攀升,超过了合格人类审稿人的容量,并延长了编辑时间线。与此同时,现代大型语言模型(LLMs)在摘要、事实核查和文献分类方面展现出令人印象深刻的能力,使得将AI整合到同行评审中越来越有吸引力——实际上,也无可避免。然而,早期的部署和非正式采用已经暴露了严重的故障模式。最近的事件表明,嵌入在稿件中的隐藏提示注入可以引导LLM生成的评审走向不合理的正面判断。补充研究还显示出对对抗性措辞、权威和长度偏见以及幻觉主张的脆弱性。这些事件引发了学术交流的一个核心问题:当AI评审科学时,我们能信任AI审稿人吗?本文提供了以安全和可靠性为中心的AI同行评审分析。我们映射了评审生命周期中的攻击——训练和数据检索、初审、深度评审、反驳和系统层面。我们通过在分层选取的ICLR 2025投稿上使用两个基于LLM的高级审稿人进行四项处理-控制探针,实例化了这一分类法,以隔离声望框架、断言强度、反驳谄媚和上下文投毒对评审分数的因果效应。总之,这一分类法和实验审计为评估和跟踪AI同行评审的可靠性提供了基于证据的基线,并突出了具体的故障点,以指导有针对性的、可测试的缓解措施。
英文摘要:
The volume of scientific submissions continues to climb, outpacing the capacity of qualified human referees and stretching editorial timelines. At the same time, modern large language models (LLMs) offer impressive capabilities in summarization, fact checking, and literature triage, making the integration of AI into peer review increasingly attractive -- and, in practice, unavoidable. Yet early deployments and informal adoption have exposed acute failure modes. Recent incidents have revealed that hidden prompt injections embedded in manuscripts can steer LLM-generated reviews toward unjustifiably positive judgments. Complementary studies have also demonstrated brittleness to adversarial phrasing, authority and length biases, and hallucinated claims. These episodes raise a central question for scholarly communication: when AI reviews science, can we trust the AI referee? This paper provides a security- and reliability-centered analysis of AI peer review. We map attacks across the review lifecycle -- training and data retrieval, desk review, deep review, rebuttal, and system-level. We instantiate this taxonomy with four treatment-control probes on a stratified set of ICLR 2025 submissions, using two advanced LLM-based referees to isolate the causal effects of prestige framing, assertion strength, rebuttal sycophancy, and contextual poisoning on review scores. Together, this taxonomy and experimental audit provide an evidence-based baseline for assessing and tracking the reliability of AI peer review and highlight concrete failure points to guide targeted, testable mitigations.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。