arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2603.09723cs.CLcs.AI
收藏

RbtAct: 用于生成可操作性评审反馈的反驳作为监督

RbtAct: Rebuttal as Supervision for Actionable Review Feedback Generation

Yale University(耶鲁大学);New York University(纽约大学);TCS Research(TCS研究院)

Sihong Wu, Yiling Ma, Yilun Zhao, Tiansheng Hu, Owen Jiang, Manasi Patwardhan, Arman Cohan

AI总结:

本文提出RbtAct,通过利用反驳作为隐式监督,优化反馈生成器以提高可操作性。引入了视角条件段级评审反馈生成任务,并构建了RMR-75K数据集,实验表明在可操作性和具体性上优于基线模型。

摘要:

大型语言模型(LLMs)越来越多地应用于科学工作流程,包括撰写同行评审报告。然而,许多AI生成的评审过于表面且不够可操作,导致作者缺乏具体的实施指导,从而产生这一工作的研究缺口。我们提出了RbtAct,旨在生成可操作的评审反馈,并将现有的同行评审反驳置于学习中心。反驳展示了哪些评审意见导致了具体的修改或特定计划,以及哪些只是被辩护。基于这一见解,我们利用反驳作为隐式监督,直接优化反馈生成器的可操作性。为支持这一目标,我们提出了一个新的任务,即视角条件段级评审反馈生成,要求模型基于完整的论文和指定的视角(如实验和写作)生成单一聚焦评论。我们还构建了一个名为RMR-75K的大型数据集,该数据集将评审段落映射到回应段落,包含视角标签和影响类别,以排序作者的采纳情况。然后,我们使用监督微调训练Llama-3.1-8B-Instruct模型,随后使用基于反驳的配对进行偏好优化。与人类专家和LLM-as-a-judge的实验表明,在可操作性和具体性上优于强基线模型,同时保持了基础和相关性。

英文摘要:

Large language models (LLMs) are increasingly used across the scientific workflow, including to draft peer-review reports. However, many AI-generated reviews are superficial and insufficiently actionable, leaving authors without concrete, implementable guidance and motivating the gap this work addresses. We propose RbtAct, which targets actionable review feedback generation and places existing peer review rebuttal at the center of learning. Rebuttals show which reviewer comments led to concrete revisions or specific plans, and which were only defended. Building on this insight, we leverage rebuttal as implicit supervision to directly optimize a feedback generator for actionability. To support this objective, we propose a new task called perspective-conditioned segment-level review feedback generation, in which the model is required to produce a single focused comment based on the complete paper and a specified perspective such as experiments and writing. We also build a large dataset named RMR-75K that maps review segments to the rebuttal segments that address them, with perspective labels and impact categories that order author uptake. We then train the Llama-3.1-8B-Instruct model with supervised fine-tuning on review segments followed by preference optimization using rebuttal derived pairs. Experiments with human experts and LLM-as-a-judge show consistent gains in actionability and specificity over strong baselines while maintaining grounding and relevance.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑