发表机构
Cornell University（康奈尔大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本研究提出VOLM框架，通过对比文档与仅由LLM从任务描述生成的文本，检测写作中的原创人类贡献，在三类文本上表现优异且对内容保留转换具鲁棒性。
AI 中文摘要
大型语言模型（LLM）已被迅速应用于各类写作任务，推动了检测LLM生成文本的工具开发。然而，这些工具大多仅测量文档表层文本由LLM生成的比例，从根本上并非为测量信息内容或想法有多少源自LLM本身，而非用户在提示中提供的内容。本研究设计了一个框架，用于衡量个人在语言模型本可轻松生成的内容之上增加了多少价值。该方法无需训练或标注数据，且从不对文档表层文本打分，使其不受风格混淆因素的影响。相反，它以不断提升的粒度提取文档内容，使用LLM从每个部分表示中重建文档，并将这些重建结果与仅从任务描述生成的重建结果进行比较。我们将此框架命名为超越语言模型的价值（Value Over Language Model, VOLM），它测量文档相对于仅由LLM从任务描述生成的替换级文档的贡献。我们在三个领域评估VOLM：新闻文章、ICLR同行评审和议论文。VOLM将人类撰写的文档与从通用任务描述生成的匹配LLM生成文档区分开来，同时对内容保留转换（包括基于LLM的重建和往返翻译）保持基本不变性。我们进一步发现，日益受限的内容提取器会减少LLM生成文本与人性化文本之间的残留差异，这证明了将信息内容与风格变化解耦的重要性。我们希望这些结果能鼓励进一步研究该框架的特定实例，并更广泛地评估LLM辅助写作中的人类贡献。
英文摘要
LLMs have been rapidly adopted across writing tasks, prompting the development of tools for detecting LLM-generated text. Yet, these tools largely measure how much of a document's surface text was written by an LLM and aren't fundamentally designed to measure how much of the information content or ideas originated from the LLM itself rather than being supplied by the user in the prompt. In this work, we design a framework that measures how much value a person adds on top of what a language model could have easily produced by itself. The method requires no training or labeled data and never scores the document's surface text, insulating it from stylistic confounders. Instead, it extracts the document's content at increasing levels of granularity, uses an LLM to reconstruct the document from each partial representation, and compares these reconstructions with those produced from the task description alone. We call this framework Value Over Language Model (VOLM), which measures a document's contribution relative to a replacement-level document that an LLM could produce from the task description alone. We evaluate VOLM with a specific instantiation of this framework across three domains: news articles, ICLR peer reviews, and argumentative essays. VOLM separates human-authored documents from matched LLM-generated documents produced from generic task descriptions, while remaining substantially invariant to content-preserving transformations, including LLM-based reconstruction and round-trip translation. We further find that increasingly constrained content extractors reduce residual differences between LLM-generated and humanized text, demonstrating the importance of disentangling informational content from stylistic variation. We hope these results encourage further work on specialized instantiations of the framework and on assessing human contributions in LLM-assisted writing more generally.
Comments39 pages
