你被告知了多少？衡量同行评审中的外部信息

中文摘要 AI 辅助

会议政策区分使用大型语言模型（LLMs）润色自己的评审与委托其进行批评，但当前的人工文本检测（ATD）方法主要衡量表面形式而非内容的来源。我们转而衡量评审所携带的外部信息：即未被评审论文和通用评审指令所解释的信息。我们提出自条件化（Self-Conditioning），一种无监督的信息论估计器，它将评审在其生成上下文下的似然性与在该上下文被从评审自身提取的提示增强后的似然性进行比较。在IntelLabs同行评审基准上，自条件化将完全委托的评审与机器润色的评审区分开来，AUC高达1.0，同时对表面改写基本不敏感。此外，随着生成器接收越来越多的外部提供信息，其得分单调地向人类评审区间移动，这与标准ATD基线不同。高温采样可以规避该估计器，但代价是输出质量下降。

英文摘要

Conference policies distinguish using Large Language Models (LLMs) to polish one's own review from delegating the critique, but current Artificial Text Detection (ATD) methods largely measure surface form rather than the origin of its content. We instead measure the external information carried by a review: information not explained by the reviewed paper and a generic reviewing instruction. We propose Self-Conditioning, an unsupervised information-theoretic estimator that compares the likelihood of a review under its production context with its likelihood when that context is augmented with hints extracted from the review itself. On the IntelLabs peer-review benchmark, Self-Conditioning separates fully-delegated from machine-polished reviews with AUC up to $1.0$ while remaining largely insensitive to surface rewriting. Moreover, as generators receive increasing amounts of externally-provided information, their scores move monotonically towards the human regime, unlike standard ATD baselines. High-temperature sampling can evade the estimator, but at the cost of output quality.

发表机构

• Sorbonne Université（索邦大学）
• CNRS（法国国家科学研究中心）
• ISIR（智能系统与机器人研究所）
• International Laboratory on Learning Systems (ILLS)（国际学习系统实验室）
• Quebec AI Institute (MILA)（魁北克人工智能研究所）
• CentraleSupélec（巴黎中央理工-高等电力学院）
• Université Paris-Saclay（巴黎萨克雷大学）

机构由 AI 辅助整理，请以论文原文为准。