arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2310.06202cs.CL
收藏

GPT-who:一种基于信息密度的机器生成文本检测器

GPT-who: An Information Density-based Machine-Generated Text Detector

The Pennsylvania State University(宾夕法尼亚州立大学);MIT Lincoln Laboratory(麻省理工学院林肯实验室)

Saranya Venkatraman, Adaku Uchendu, Dongwon Lee

AI总结:

本文提出受心理语言学启发、基于统一信息密度特征的领域无关统计检测器GPT-who,用于识别LLM生成文本,并在4个基准上显著优于现有检测器。

摘要:

统一信息密度(UID)原则认为,人类在语言生成过程中倾向于均匀分布信息。我们考察该UID原则是否有助于捕捉大型语言模型(LLM)生成文本与人类生成文本之间的差异。我们提出GPT-who,这是首个受心理语言学启发、与领域无关的统计检测器。该检测器采用基于UID的特征,对每个LLM和人类作者的独特统计特征进行建模,以实现准确检测。我们使用4个大规模基准数据集评估该方法,发现GPT-who在跨领域场景中比GLTR、GPTZero、DetectGPT、OpenAI detector和ZeroGPT等最先进检测器(包括统计型与非统计型)高出20%以上。除性能更优外,它计算成本低,并使用可解释的文本表示。我们发现,即使表层文本难以辨别,GPT-who仍能区分由非常复杂的LLM生成的文本。所有数据集的基于UID的度量和代码可在https://github.com/saranya-venkatraman/gpt-who获取。

英文摘要:

The Uniform Information Density (UID) principle posits that humans prefer to spread information evenly during language production. We examine if this UID principle can help capture differences between Large Language Models (LLMs)-generated and human-generated texts. We propose GPT-who, the first psycholinguistically-inspired domain-agnostic statistical detector. This detector employs UID-based features to model the unique statistical signature of each LLM and human author for accurate detection. We evaluate our method using 4 large-scale benchmark datasets and find that GPT-who outperforms state-of-the-art detectors (both statistical- & non-statistical) such as GLTR, GPTZero, DetectGPT, OpenAI detector, and ZeroGPT by over $20$% across domains. In addition to better performance, it is computationally inexpensive and utilizes an interpretable representation of text articles. We find that GPT-who can distinguish texts generated by very sophisticated LLMs, even when the overlying text is indiscernible. UID-based measures for all datasets and code are available at https://github.com/saranya-venkatraman/gpt-who.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑