FactReview：基于执行式声明验证的证据驱动同行评审
FactReview: Evidence-Grounded Peer Review with Execution-Based Claim Verification
AI总结：
提出FactReview系统，通过提取与评审相关的声明、将其与相关工作关联，并在代码可用时在固定修复预算下执行发布工件来审计经验声明，覆盖84%的声明，将评审质量提升至4.86/5，并将评审时间减少58%。
摘要：
基于LLM的评审系统通常仅以手稿为输入，使得文献和基于代码的声明难以验证。我们提出FactReview，一个提取与评审相关的声明、将其与相关工作关联，并在代码可用时在固定修复预算下执行发布工件以审计经验声明的系统。在35篇ML论文和463个基准主要声明中，FactReview覆盖了84%的声明。在证据感知评分标准下，其评审在整体质量上得分为4.86/5，比DeepReview-v2高0.7，比匹配的OpenReview评论高1.5。移除执行证据会改变17%的声明状态，超过任何其他单一证据来源。在一项评审辅助研究中，FactReview将平均评审时间减少了58%，同时将基准声明覆盖率从87%提高到99%。我们认为LLM评审者应审计经验声明，而非做出接受或拒绝的决定。代码公开于：https://github.com/DEFENSE-SEU/FactReview。
英文摘要：
Large language model (LLM)-based reviewing systems typically assess manuscripts in isolation, leaving literature- and code-dependent claims difficult to verify. We present FactReview, an audit pipeline that extracts review-relevant claims, grounds them in related work and reference checks, and, when code is available, executes released artifacts under a fixed repair budget. On 26 paper-disjoint test papers with 354 human-verified claims, FactReview achieves 84.3\% F1 for claim recovery. In a same-backend, evidence-matched comparison, FactReview scores 4.72/5 overall, outperforming a direct LLM reviewer by 0.74 points. Removing execution evidence changes 17.0\% of claim statuses, more than removing any other single evidence source. In a reviewer-assistance study, FactReview reduces mean review time by 58\% while increasing benchmark-claim coverage from 87\% to 99\%. FactReview supports evidence-based claim auditing, with acceptance decisions reserved for human reviewers. The code is public at https://github.com/DEFENSE-SEU/FactReview.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
