发表机构
Pennsylvania State University; University of Sheffield（宾夕法尼亚州立大学; 谢菲尔德大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
针对LLM辅助评审中洞察力归属问题，提出Insight Provenance任务及InsightProv-v0数据集，并设计两阶段对抗框架抑制捷径，发现AI洞察多依赖通用信息而人类更引入外部知识，表明想法溯源比措辞更具持久信号。
AI 中文摘要
随着大语言模型（LLM）越来越多地辅助科学写作和同行评审，检测文本由谁撰写已不再足够：我们需要确定潜在洞察力的贡献者。我们引入了“洞察力溯源”（Insight Provenance）任务，即识别评审洞察力是源自人类、LLM，还是两者的混合贡献。我们从4,057篇科学论文和12,660条人类评审中构建了InsightProv-v0数据集，使用GPT-4o、Gemini和DeepSeek模拟不同水平的LLM参与度，并在句子级别标注溯源信息。我们表明，在原始数据上的强性能可能具有误导性，因为模型会利用语言和文本作者身份捷径，这些捷径在逐步去偏的评估下会大幅退化。因此，我们提出了一种两阶段对抗性框架，在抑制捷径信号的同时保留与溯源相关的信息。在检测之外，广泛的分析揭示了是什么使得智力作者身份可被识别：论文依据和相邻评审上下文提供了互补的溯源信号，而人类、混合和AI洞察力在信息来源和失败模式上系统性地存在差异。最引人注目的是，AI洞察力主要停留在通用或论文提供的信息附近，而人类洞察力更常引入外部知识和独立判断。这些发现表明，虽然措辞可以由LLM重写，但想法的溯源留下了更深层且更持久的信号。
英文摘要
As LLMs increasingly assist scientific writing and peer review, detecting who wrote the text is no longer sufficient: we need to determine who contributed the underlying insight. We introduce Insight Provenance, the task of identifying whether a review insight originates from a human, an LLM, or their hybrid contribution. We construct InsightProv-v0 from 4,057 scientific papers and 12,660 human reviews, simulating different levels of LLM involvement with GPT-4o, Gemini, and DeepSeek and annotating provenance at the sentence level. We show that strong performance on raw data can be misleading, as models exploit linguistic and textual-authorship shortcuts that degrade substantially under progressively debiased evaluation. We therefore propose a two-stage adversarial framework that suppresses shortcut signals while preserving provenance-relevant information. Beyond detection, extensive analyses reveal what makes intellectual authorship identifiable: paper grounding and neighboring review context provide complementary provenance signals, while human, hybrid, and AI insights systematically differ in their information sources and failure modes. Most strikingly, AI insights predominantly remain close to generic or paper-provided information, whereas human insights more often introduce external knowledge and independent judgment. These findings suggest that while wording can be rewritten by an LLM, the provenance of an idea leaves a deeper and more persistent signal.
