ReviewGuard: 使LLM辅助的同行评审与长期科学影响力对齐
ReviewGuard: Aligning LLM-Assisted Peer Review with Long-Term Scientific Impact
AI总结：
提出ReviewGuard两阶段框架，通过强化学习将LLM生成的评审与基于引用的长期影响力对齐，在2万篇论文上优于人类评审和监督模型，高影响力论文识别率提升5.6倍。
摘要：
同行评审是科学质量控制的核心，但它可能低估后来获得显著引用影响力的论文。虽然前沿的大语言模型在自动化同行评审方面显示出前景，但它们主要模仿人类评审者的偏好，而非预测长期科学价值。我们引入了ReviewGuard，一个两阶段框架，将LLM生成的评审与基于引用的长期科学影响力估计对齐，而非与同期评审者的判断对齐。在来自OpenReview的20,861篇AI/ML论文上，结合Semantic Scholar引用数据，ReviewGuard在已被拒后发表的论文上，其未来引用的Spearman相关系数达到ρ=0.776，优于人类评审者（ρ=0.492）和监督专家模型（ρ=0.681）。在相同决策阈值下，ReviewGuard标记了10.2%的高影响力被拒论文，而人类评审者仅为1.8%，对应5.6倍的提升。我们的结果表明，影响力对齐的强化学习可以为编辑提供识别高潜力工作的补充信号，而不取代人类判断。
英文摘要：
Peer review is central to scientific quality control, yet it can undervalue papers that later achieve substantial citation impact. While frontier large language models have shown promise in automating aspects of peer review, they primarily mimic human reviewer preferences rather than predict long-term scientific value. We introduce ReviewGuard, a two-stage framework that aligns LLM-generated reviews with citation-based estimates of long-term scientific impact rather than contemporaneous reviewer judgments. On 20,861 AI/ML papers from OpenReview augmented with Semantic Scholar citation data, ReviewGuard achieves a Spearman correlation of \r{ho} = 0.776 with future citations on rejected-then-published papers, outperforming human reviewers (\r{ho} = 0.492) and a supervised Expert model (\r{ho} = 0.681). Under the same decision threshold, ReviewGuard flags 10.2% of high-impact rejected papers, compared with 1.8% for human reviewers, corresponding to a 5.6x improvement. Our results demonstrate that impact-aligned reinforcement learning can provide editors with a complementary signal for identifying high-potential work, without replacing human judgment.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
