发表机构
Shandong University; Huawei Technologies Co., Ltd.（山东大学; 华为技术有限公司）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
RubricReviewer是一种基于评分规则驱动的框架，通过将评分规则生成为中间步骤，并结合Scout与Aligner模型，生成更全面、具判别性且抗攻击的评审意见。
AI 中文摘要
主要学术会议的同行评审正面临前所未有的投稿压力，这促使人们将大语言模型（LLMs）用作评审助手。然而，现有的基于LLM的评审者存在两个结构性局限：其一，它们将稿件直接映射至评审意见，使底层评分规则（rubric）隐含，且将评分规则的推导与判断纠缠在一起；其二，主流范式仅能捕捉优质评审的一半内容：无训练的智能体（agent）可收集广泛证据，但会产生无方向的批评；基于训练的评审者则继承了人类的判别性判断，同时也继承了其噪声与覆盖不均的问题。我们提出RubricReviewer，这是一个完全基于评分规则驱动的框架，可解决上述两个局限。该框架将评分规则生成为明确的中间步骤，使评审生成与最终评估均以适配稿件的评分规则为条件；它还进一步结合了收集外部证据的无训练智能体Scout，以及利用该证据的人类对齐的训练模型Aligner，融合了两种监督源的优势。对真实投稿的实验表明，RubricReviewer生成的评审意见比现有系统明显更全面、更具判别性，且对对抗性提示注入攻击表现出最强的鲁棒性； ablation研究进一步证实了每个组件的必要性。
英文摘要
Peer review at major venues is under unprecedented submission pressure, motivating the use of large language models (LLMs) as review assistants. Existing LLM-based reviewers, however, face two structural limitations. First, they map manuscripts directly to reviews, leaving the underlying rubric implicit and entangling its derivation with the judgement. Second, the prevailing paradigms each capture only half of a good review: training-free agents gather broad evidence but produce undirected critiques, while training-based reviewers inherit human discriminative judgement together with its noise and uneven coverage. We introduce RubricReviewer, a fully rubric-driven framework that addresses both limitations. It makes rubric generation an explicit intermediate step, so that both review generation and the final assessment are conditioned on paper-adaptive rubrics. It further combines a training-free agent (Scout) that gathers external evidence with a human-aligned trained model (Aligner) that consumes this evidence, fusing the strengths of both supervision sources. Experiments on real-world submissions show that RubricReviewer produces reviews that are markedly more comprehensive and more discriminative than prior systems, and exhibits the strongest robustness against adversarial prompt-injection attacks. Ablation studies further confirm the necessity of each component.
