arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2401.12970cs.CL
收藏

Raidar:通过重写进行生成式AI检测(geneRative AI Detection viA Rewriting)

Raidar: geneRative AI Detection viA Rewriting

Columbia University(哥伦比亚大学);Rutgers University(罗格斯大学)

Chengzhi Mao, Carl Vondrick, Hao Wang, Junfeng Yang

AI总结:

本研究发现大语言模型重写时对AI生成文本修改更少,据此提出Raidar检测方法,通过计算重写编辑距离检测AI内容,多领域下F1分数最高提升29点,兼容黑盒模型且鲁棒性强。

摘要:

我们发现,当执行重写任务时,大语言模型(LLM)更倾向于修改人类撰写的文本,而非AI生成的文本。这种倾向的产生是因为LLM通常认为AI生成的文本质量更高,因此做出的修改更少。我们提出了一种检测AI生成内容的方法:通过提示LLM重写文本并计算输出的编辑距离来实现检测。我们将这种“通过重写进行生成式AI检测(geneRative AI Detection viA Rewriting)”的方法命名为Raidar。在新闻、创意写作、学生论文、代码、Yelp评论和arXiv论文等多个领域中,Raidar显著提升了现有AI内容检测模型(包括学术模型和商业模型)的F1检测分数,提升幅度最高可达29个点。我们的方法仅基于词符号运行,无需高维特征,可兼容黑盒LLM,且对新内容具有固有鲁棒性。我们的研究结果通过机器自身的视角,揭示了机器生成文本的独特印记。

英文摘要:

We find that large language models (LLMs) are more likely to modify human-written text than AI-generated text when tasked with rewriting. This tendency arises because LLMs often perceive AI-generated text as high-quality, leading to fewer modifications. We introduce a method to detect AI-generated content by prompting LLMs to rewrite text and calculating the editing distance of the output. We dubbed our geneRative AI Detection viA Rewriting method Raidar. Raidar significantly improves the F1 detection scores of existing AI content detection models -- both academic and commercial -- across various domains, including News, creative writing, student essays, code, Yelp reviews, and arXiv papers, with gains of up to 29 points. Operating solely on word symbols without high-dimensional features, our method is compatible with black box LLMs, and is inherently robust on new content. Our results illustrate the unique imprint of machine-generated text through the lens of the machines themselves.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑