arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.28054cs.CL
收藏

谁写了这本书?检测与归因LLM代笔作者

Who Wrote the Book? Detecting and Attributing LLM Ghostwriters

School of Computing and Information Systems, The University of Melbourne, Australia(墨尔本大学计算与信息学院);School of Computing, FSE, Macquarie University, Australia(麦考瑞大学计算学院)

Anudeex Shetty, Qiongkai Xu, Olga Ohrimenko, Jey Han Lau

AI总结:

提出GhostWriteBench数据集和TRACE指纹方法,用于检测和归因LLM生成的长文本,在跨域和未见模型上达到SOTA性能。

摘要:

在本文中,我们介绍了GhostWriteBench,一个用于LLM作者归因的数据集。它包含由前沿LLM生成的长篇文本(每本书超过5万字),旨在测试跨多个分布外(OOD)维度的泛化能力,包括领域和未见过的LLM作者。我们还提出了TRACE——一种新颖的、可解释且轻量级的指纹方法——适用于开源和闭源模型。TRACE通过捕获由另一个轻量级语言模型估计的token级转换模式(例如,词排名)来创建指纹。在GhostWriteBench上的实验表明,TRACE实现了最先进的性能,在OOD设置中保持鲁棒性,并且在有限训练数据场景下表现良好。

英文摘要:

In this paper, we introduce GhostWriteBench, a dataset for LLM authorship attribution. It comprises long-form texts (50K+ words per book) generated by frontier LLMs, and is designed to test generalisation across multiple out-of-distribution (OOD) dimensions, including domain and unseen LLM author. We also propose TRACE -- a novel fingerprinting method that is interpretable and lightweight -- that works for both open- and closed-source models. TRACE creates the fingerprint by capturing token-level transition patterns (e.g., word rank) estimated by another lightweight language model. Experiments on GhostWriteBench demonstrate that TRACE achieves state-of-the-art performance, remains robust in OOD settings, and works well in limited training data scenarios.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑