arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2501.02406stat.MLcs.AIcs.CLcs.ITcs.LGmath.IT
收藏

无需训练的LLM文本归因方法

A Training-free Method for LLM Text Attribution

Ross School of Business, University of Michigan, United States(密歇根大学罗斯商学院);Department of Statistics, University of Michigan, United States(密歇根大学统计学系)

Tara Radvand, Izak Duenyas, Ambuj Tewari

AI总结:

本文提出无需训练的LLM文本归因方法,通过零样本统计测试区分不同LLM生成文本,并证明测试误差随文本长度指数下降,同时验证理论结果和对抗性后编辑的鲁棒性。

摘要:

验证内容来源对于许多组织的运作至关重要,例如教育机构、社交媒体平台和公司。随着大型语言模型(LLM)生成的文本几乎无法与人类生成的内容区分开来,这一问题变得越来越具有挑战性。此外,许多机构使用内部LLM,并希望确保外部、非授权的LLM不在其机构内生成内容。本文回答了以下问题:给定一段文本,能否确定其是否由特定LLM生成,同时保证极低的误报率?我们将LLM文本建模为一个具有完全历史依赖性的序列随机过程。然后,我们设计了零样本统计测试,以(i)区分由两个不同已知LLM集合A(非授权)和B(内部)生成的文本,以及(ii)确定文本是否由已知LLM或任何未知模型生成。我们证明了我们的测试的I型和II型错误随文本长度呈指数下降。我们还扩展了我们的理论到通过采样进行的黑盒访问,并表征了获得与白盒设置(即具有访问A)相同I型和II型错误上界所需的样本量。我们通过提供信息论下界来证明我们的上界紧致性。接下来,我们展示了数值实验以验证我们的理论结果,并评估其在对抗性后编辑设置中的鲁棒性。我们的工作在需要确定文本来源的重要实践中具有广泛的应用,也可用于对抗虚假信息并确保符合新兴AI法规。见https://github.com/TaraRadvand74/llm-text-detection 以获取代码、数据和项目在线演示。

英文摘要:

Verifying the provenance of text is increasingly important for firms, educational institutions, and online platforms as Large Language Models (LLMs) produce output that is nearly indistinguishable from human-generated content. We study the problem of determining whether a given text was generated by a particular LLM while controlling the false positive rate. We model LLM-generated text as a sequential stochastic process and develop training-free statistical tests to (i) distinguish between text produced by two known sets of LLMs and (ii) determine whether text was generated by a known LLM or by a distinguishable unknown source, such as a human or another model. We prove that both Type I and Type II errors decay exponentially with text length, establish analogous guarantees for black-box access via sampling, and provide an information-theoretic lower bound showing that there exist model pairs for which no statistical test can make both errors decay faster than exponentially with text length. Numerical experiments empirically evaluate the tests in practical settings and demonstrate strong overall performance, including under many adversarial edits. Our framework provides rigorous guarantees for LLM provenance detection, with applications to content verification, institutional compliance, and misinformation mitigation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑