Scholarly publishing requires scalable scrutiny supported by auditable evidence. This paper presents a two-component benchmark of xPeer, the peer-review simulation engine delivered through the xPeerd.com web front-end. The operational component analyzes 352 of 500 simulation records retained under stable-task criteria across disciplines and review modes. The human-reference component releases 1,108 version-1 F1000Research manuscript records with linked human reports and applies a prespecified two-human/two-xPeer comparison. Human-review text and recommendations remained outside the generation input, and source joining occurred after xPeer outputs had been persisted. This procedure defines workflow-level review withholding; prior model exposure falls outside the recorded design. Among 802 records with exactly two human reports, 271 contained two usable xPeer reviewer fields, giving a complete-pair availability rate of 33.8%. Under deterministic extraction rules, median manuscript-level report length was 1,889 words for xPeer and 763 for humans, while median concern count was 41 and 13, respectively. xPeer reports showed higher targeting, category coverage, and executability. Human reports showed higher explicit-reasoning language, lexical manuscript attestation, taxonomy-based scientific relevance, and lower mean within-source redundancy. Cross-source lexical concern matching and recommendation agreement were low. The evidence therefore defines distinct observable review profiles and a transparent reproducibility baseline. Scientific correctness of individual concerns, autonomous editorial use, and cross-system superiority require expert adjudication and common-protocol testing. The study-level dataset and exact version-pinned reproducibility record are archived on Zenodo
MOPRD: A multidisciplinary open peer review dataset
School of Informatics, Xiamen University(厦门大学信息学院);Key Laboratory of Digital Protection and Intelligent Processing of Intangible Cultural Heritage of Fujian and Taiwan, Ministry of Culture and Tourism(文化和旅游部闽台非物质文化遗产数字化保护与智能处理重点实验室);College of Foreign Languages and Cultures, Xiamen University(厦门大学外文学院)
FMMD: A multimodal multidisciplinary dataset of open peer reviews from F1000Research
School of Computer Science and Engineering, Guangzhou Institute of Science and Technology(计算机科学与工程学院,广州科学与技术研究所);College of Foreign Languages and Cultures, Xiamen University(外语学院,厦门大学);Science and Education Evaluation Lab, Guangzhou Institute of Science and Technology(科学与教育评估实验室,广州科学与技术研究所)
Are We There Yet? Revealing the Risks of Utilizing Large Language Models in Scholarly Peer Review
Shanghai Jiao Tong University(上海交通大学);Georgia Institute of Technology(佐治亚理工学院);Shanghai AI Laboratory(上海人工智能实验室);University of Georgia(佐治亚大学);Oxford University(牛津大学)
Automated scholarly paper review: Concepts, technologies, and challenges
Xiamen University(厦门大学);Key Laboratory of Digital Protection and Intelligent Processing of Intangible Cultural Heritage of Fujian and Taiwan, Ministry of Culture and Tourism(文化和旅游部闽台非物质文化遗产数字化保护与智能处理重点实验室)
CoCoNUTS: Concentrating on Content while Neglecting Uninformative Textual Styles for AI-Generated Peer Review Detection
Chinese Information Processing Laboratory, Institute of Software, Chinese Academy of Sciences(中国科学院软件研究所中文信息处理实验室);University of Chinese Academy of Sciences(中国科学院大学)
Large language models for automated scholarly paper review: A survey
School of Computer Science and Engineering, Guangzhou Institute of Science and Technology(计算机科学与工程学院,科学与技术研究院);School of Economics and Management, Guizhou Normal University(经济学与管理学院,贵州师范大学);School of Artificial Intelligence, Guangzhou Institute of Science and Technology(人工智能学院,科学与技术研究院)
Pre-review to Peer review: Pitfalls of Automating Reviews using Large Language Models
AllSci Corp(AllSci公司);Sunwater Capital(Sunwater资本);Kellogg School of Management, Northwestern University(西北大学凯洛格管理学院);Dept. of Computer Science, Northern Illinois University(北伊利诺伊大学计算机科学系);Institute of Foundation Models, MBZUAI(基础模型研究所,MBZUAI)