发表机构
Georgia Institute of Technology; Microsoft（佐治亚理工学院; 微软公司）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本文推出Metag数据集，用于构建元评审智能体，助力元评审员快速确认作者对评审意见的回应及变更位置，提升同行评审透明度与可追溯性。
AI 中文摘要
AI工具越来越多地支持科研全流程任务，从实验设计、稿件准备到同行评审。与此同时，会议投稿量的持续增长加重了元评审员的负担，元评审员必须综合评审意见、作者反驳内容及稿件修改情况。为解决这一问题，本文推出Metag，这是一个用于加速元评审智能体开发的数据集，专门用于识别稿件在评审-反驳过程中发生的变更。每个实例包含评审意见、作者提出的解决方案以及实现所述变更的稿件差异。Metag通过获取评审截止前和录用后的稿件版本，计算两者差异，并请人工标注员将这些差异与OpenReview讨论中的行动项对齐来构建。该数据集包含349个与稿件差异关联的高质量行动项，将支持开发方法，帮助元评审员快速确认作者是否回应了评审意见以及这些变更在稿件中的位置，从而提升同行评审的透明度和可追溯性。该数据集可在此httpsURL获取。
英文摘要
AI tools increasingly support tasks across the scientific research cycle, from experiment design and manuscript preparation to peer review. At the same time, the continuing growth in conference submissions has increased the burden on meta-reviewers, who must synthesize reviewer feedback, author rebuttals, and manuscript revisions. To address this concern, this paper introduces Metag, a dataset to accelerate the development of meta-reviewing agents, specifically to identify changes made to scientific articles during the review-rebuttal process. Each instance contains a reviewer concern, the author's proposed resolution, and the manuscript diffs implementing the stated change. Metag is collected by obtaining manuscript versions from before the review deadline and after acceptance, computing differences between the two documents, and asking human annotators to align these differences with action items from OpenReview discussions. The resulting dataset consists of 349 high-quality action items tied to paper differences and will enable building methods to empower meta reviewers to quickly identify whether authors have addressed reviewer statements and where in the paper those changes have been made, resulting in additional transparency and traceability throughout peer review. The dataset is publicly available at https://github.com/microsoft/Metag-dataset.
Comments23 pages, 5 figures, 6 tables
