基于LLM的科学同行评审：方法、基准与可靠性挑战
LLM-Based Scientific Peer Review: Methods, Benchmarks, and Reliability Challenges
AI总结：
本文综述了基于大语言模型的科学同行评审，聚焦批评生成与分数预测两大功能，分析了建模方法、基准评估、数据局限及鲁棒性风险，并提出了未来研究方向。
摘要：
科学投稿的快速增长将传统同行评审推向其可扩展性的极限，促使人们探索大语言模型作为智能自动化评估助手。尽管近期研究表明，LLM能够生成流畅的批评并近似评审者分数，但其作为决策支持系统的可靠性、鲁棒性和安全性仍未被充分理解。本综述对基于LLM的科学同行评审进行了系统级分析，聚焦两个核心评估功能：批评生成和分数预测。我们提出了建模方法的结构化分类（包括基于提示、监督、检索增强和对齐优化的方法），并综合了现有基准的实证发现。我们分析了限制当前评估实践的数据集约束、评估缺陷和领域集中偏差。除了性能指标，我们识别了新兴的鲁棒性风险，包括提示注入、数据投毒、检索漏洞和奖励黑客，这些风险使自动化评审流程面临策略性操纵。从数据挖掘角度，我们概述了建模主观分歧和跨领域泛化的关键开放挑战。通过将自动化同行评审重新定义为高风险、多目标决策问题，本综述为开发鲁棒、透明且可信赖的AI辅助科学评估系统提供了路线图。
英文摘要：
The rapid growth of scientific submissions has pushed traditional peer review toward its scalability limits, motivating the exploration of large language models (LLMs) as intelligent automated evaluation assistants. Although recent studies show that LLMs can generate fluent critiques and approximate reviewer scores, their reliability, robustness, and security as decision-support systems remain insufficiently understood. This survey offers a systems-level analysis of LLM-based scientific peer review, focusing on two core evaluative functions: critique generation and score prediction. We present a structured taxonomy of modeling approaches (including prompt-based, supervised, retrieval-augmented, and alignment-optimized approaches), and synthesize empirical findings across existing benchmarks. We analyze dataset constraints, evaluation shortcomings, and domain concentration biases that limit current assessment practices. Beyond performance metrics, we identify emerging robustness risks, including prompt injection, data poisoning, retrieval vulnerabilities, and reward hacking, which expose automated review pipelines to strategic manipulation. From a data mining perspective, we outline key open challenges in modeling subjective disagreement and cross-domain generalization. By reframing automated peer review as a high-stakes, multi-objective decision problem, this survey provides a roadmap for developing robust, transparent, and trustworthy AI-assisted scientific evaluation systems.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
