解读文本真实性：透过大语言语义视角识别人类与机器生成文本的通用策略
Deciphering Textual Authenticity: A Generalized Strategy through the Lens of Large Language Semantics for Detecting Human vs. Machine-Generated Text
AI总结：
针对真实场景中多生成器、多领域机器文本检测泛化不足的问题，提出 T5LLMCipher，融合预训练 T5 编码器与 LLM 嵌入子聚类，显著提升跨域 F1 并准确识别生成器。
摘要：
随着大型语言模型（LLM）近期的迅速普及，对检测机器生成文本的工具需求日益增长。有效检测机器生成文本面临两个相关问题：首先，这些方法在面向真实场景时的泛化能力严重受限，因为真实场景中的机器生成文本由多种生成器产生，包括但不限于 GPT-4 和 Dolly，并跨越从学术手稿到社交媒体帖子等多种领域。其次，现有检测方法通过受限的二分类视角看待 LLM 生成的文本，忽视了不同 LLM 所生成产物的细微差异。本文对真实场景中的机器生成文本检测进行了系统研究。我们首先研究了最先进方法的有效性，发现它们在面对真实世界中由多样生成器和领域产生的文本时存在严重局限。此外，对预训练 LLM 编码器所得嵌入进行的 t-SNE 可视化表明，这些嵌入无法可靠区分人类文本与机器生成文本。基于这些发现，我们提出了一个名为 T5LLMCipher 的新型系统，用于检测机器生成文本；该系统将预训练 T5 编码器与 LLM 嵌入子聚类相结合，以应对真实世界中由多样生成器和领域产生的文本。我们在 9 个机器生成文本系统和 9 个领域上评估了所提方法，结果表明该方法具备最先进的泛化能力：与表现最佳的现有方法相比，在未见过的生成器和领域上，机器生成文本的 F1 分数平均提升 19.6%，并且能够以 93.6% 的准确率正确归属文本的生成器。
英文摘要：
With the recent proliferation of Large Language Models (LLMs), there has been an increasing demand for tools to detect machine-generated text. The effective detection of machine-generated text face two pertinent problems: First, they are severely limited in generalizing against real-world scenarios, where machine-generated text is produced by a variety of generators, including but not limited to GPT-4 and Dolly, and spans diverse domains, ranging from academic manuscripts to social media posts. Second, existing detection methodologies treat texts produced by LLMs through a restrictive binary classification lens, neglecting the nuanced diversity of artifacts generated by different LLMs. In this work, we undertake a systematic study on the detection of machine-generated text in real-world scenarios. We first study the effectiveness of state-of-the-art approaches and find that they are severely limited against text produced by diverse generators and domains in the real world. Furthermore, t-SNE visualizations of the embeddings from a pretrained LLM's encoder show that they cannot reliably distinguish between human and machine-generated text. Based on our findings, we introduce a novel system, T5LLMCipher, for detecting machine-generated text using a pretrained T5 encoder combined with LLM embedding sub-clustering to address the text produced by diverse generators and domains in the real world. We evaluate our approach across 9 machine-generated text systems and 9 domains and find that our approach provides state-of-the-art generalization ability, with an average increase in F1 score on machine-generated text of 19.6\% on unseen generators and domains compared to the top performing existing approaches and correctly attributes the generator of text with an accuracy of 93.6\%.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
