arXivDaily arXiv每日学术速递 周一至周五更新
北美退伍军人节(Veterans Day),12 日无更新
arXiv 2304.04736cs.CLcs.AIcs.LG
收藏

论AI生成文本检测的可能性

On the Possibilities of AI-Generated Text Detection

University of Maryland(马里兰大学)

Souradip Chakraborty, Amrit Singh Bedi, Sicheng Zhu, Bang An, Dinesh Manocha, Furong Huang

AI总结:

本文从信息论角度证明AI生成文本在分布可分时可检测,给出样本复杂度界,并通过多数据集、多生成器与检测器实验验证多样本检测的可行性。

摘要:

我们的工作解决了区分大语言模型(LLM)生成文本与人类生成文本这一关键问题,该任务对众多应用都至关重要。尽管关于这种区分的可行性仍存在争论,我们提出证据支持其可以持续实现,唯一例外是当人类文本与机器文本的分布在整个支撑集上都不可区分时。基于信息论,我们认为,随着机器生成文本接近人类水平的质量,检测所需的样本量会增加。我们建立了检测AI生成文本的精确样本复杂度界,为未来开发先进的多样本检测器奠定基础。我们在多个数据集(Xsum、Squad、IMDb和Kaggle FakeNews)上的实证评估证实了增强检测方法的可行性。我们测试了多种最先进的文本生成器,包括GPT-2、GPT-3.5-Turbo、Llama、Llama-2-13B-Chat-HF和Llama-2-70B-Chat-HF,并对照包括oBERTa-Large/Base-Detector、GPTZero在内的检测器。我们的发现与OpenAI关于序列长度的实证数据一致,首次为这些观察结果提供了理论依据。

英文摘要:

Our work addresses the critical issue of distinguishing text generated by Large Language Models (LLMs) from human-produced text, a task essential for numerous applications. Despite ongoing debate about the feasibility of such differentiation, we present evidence supporting its consistent achievability, except when human and machine text distributions are indistinguishable across their entire support. Drawing from information theory, we argue that as machine-generated text approximates human-like quality, the sample size needed for detection increases. We establish precise sample complexity bounds for detecting AI-generated text, laying groundwork for future research aimed at developing advanced, multi-sample detectors. Our empirical evaluations across multiple datasets (Xsum, Squad, IMDb, and Kaggle FakeNews) confirm the viability of enhanced detection methods. We test various state-of-the-art text generators, including GPT-2, GPT-3.5-Turbo, Llama, Llama-2-13B-Chat-HF, and Llama-2-70B-Chat-HF, against detectors, including oBERTa-Large/Base-Detector, GPTZero. Our findings align with OpenAI's empirical data related to sequence length, marking the first theoretical substantiation for these observations.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑