PeerPrism：同行评价专业性与审稿写作AI
PeerPrism: Peer Evaluation Expertise vs Review-writing AI
AI总结：
本文提出PeerPrism基准测试，通过20690篇同行评审数据，区分思想来源与文本来源，揭示现有检测方法在混合场景下的不足，强调需多维度建模作者身份。
摘要：
大型语言模型（LLMs）在科学同行评审中被广泛用于协助草稿、重写、扩展和细化。然而，现有同行评审LLM检测方法大多将作者身份视为二元问题（人类vs. AI），而未考虑现代评审流程的混合性质。在实践中，评价思想和表面实现可能来自不同来源，形成人类-AI协作的光谱。本文引入PeerPrism，一个包含20690篇同行评审的大型基准测试，专门用于区分思想来源与文本来源。我们构建了涵盖完全人类、完全合成和多种混合转换的受控生成制度。此设计使系统评估检测器是否能识别表面文本的来源或评价推理的来源。我们对最先进的LLM文本检测方法在PeerPrism上进行了基准测试。尽管某些方法在标准二元任务（人类vs.完全合成）上表现出高准确性，但其预测在混合制度下显著偏离。特别是当思想来源于人类但表面文本由AI生成时，检测器经常产生分歧和矛盾分类。伴随风格学和语义分析，我们的结果表明，当前检测方法将表面实现与智力贡献混淆。总体而言，我们证明LLM在同行评审中的检测不能简化为二元归属问题。相反，作者身份必须建模为涵盖语义推理和风格实现的多维结构。PeerPrism是首个评估这些场景下人类-AI协作的基准测试。我们释放所有代码、数据、提示和评估脚本以促进可重复研究，详见https://github.com/Reviewerly-Inc/PeerPrism。
英文摘要：
Large Language Models (LLMs) are increasingly used in scientific peer review, assisting with drafting, rewriting, expansion, and refinement. However, existing peer-review LLM detection methods largely treat authorship as a binary problem-human vs. AI-without accounting for the hybrid nature of modern review workflows. In practice, evaluative ideas and surface realization may originate from different sources, creating a spectrum of human-AI collaboration. In this work, we introduce PeerPrism, a large-scale benchmark of 20,690 peer reviews explicitly designed to disentangle idea provenance from text provenance. We construct controlled generation regimes spanning fully human, fully synthetic, and multiple hybrid transformations. This design enables systematic evaluation of whether detectors identify the origin of the surface text or the origin of the evaluative reasoning. We benchmark state-of-the-art LLM text detection methods on PeerPrism. While several methods achieve high accuracy on the standard binary task (human vs. fully synthetic), their predictions diverge sharply under hybrid regimes. In particular, when ideas originate from humans but the surface text is AI-generated, detectors frequently disagree and produce contradictory classifications. Accompanied by stylometric and semantic analyses, our results show that current detection methods conflate surface realization with intellectual contribution. Overall, we demonstrate that LLM detection in peer review cannot be reduced to a binary attribution problem. Instead, authorship must be modeled as a multidimensional construct spanning semantic reasoning and stylistic realization. PeerPrism is the first benchmark evaluating human-AI collaboration in these settings. We release all code, data, prompts, and evaluation scripts to facilitate reproducible research at https://github.com/Reviewerly-Inc/PeerPrism.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
