arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2402.11167cs.CLcs.AI
收藏

ToBlend:利用LLM集成进行Token级混合以攻击AI生成文本检测

ToBlend: Token-Level Blending With an Ensemble of LLMs to Attack AI-Generated Text Detection

Indiana University Bloomington(印第安纳大学布卢明顿分校)

Fan Huang, Haewoon Kwak, Jisun An

AI总结:

提出ToBlend这一token级集成文本生成方法,通过从多个候选LLMs中随机采样token,有效降低主流AI文本检测性能,并构建微调Llama3.1模型提升识别准确度。

摘要:

AI内容检测模型对复杂对抗策略(如释义或词语替换)的鲁棒性是自然语言生成(NLG)应用中日益受到关注的问题。本研究提出ToBlend,一种新颖的token级集成文本生成方法,通过利用多组候选生成式大型语言模型(LLMs)来挑战当前AI内容检测方法的鲁棒性。通过从候选LLMs集合中随机采样token,我们发现ToBlend显著降低了大多数主流AI内容检测方法的性能。我们基于经验丰富的人类专家的标注,评估了不同ToBlend设置下生成的文本质量。我们提出了一个微调的Llama3.1模型,以更准确地区分ToBlend生成的文本。我们的发现强调了我们提出的文本生成方法在欺骗和改进检测模型方面的巨大潜力。我们的数据集、代码和标注已开源。

英文摘要:

The robustness of AI-content detection models against sophisticated adversarial strategies, such as paraphrasing or word switching, is a rising concern in natural language generation (NLG) applications. This study proposes ToBlend, a novel token-level ensemble text generation method to challenge the robustness of current AI-content detection approaches by utilizing multiple sets of candidate generative large language models (LLMs). By randomly sampling token(s) from candidate LLMs sets, we find ToBlend significantly drops the performance of most mainstream AI-content detection methods. We evaluate the text quality produced under different ToBlend settings based on annotations from experienced human experts. We proposed a fine-tuned Llama3.1 model to distinguish the ToBlend generated text more accurately. Our findings underscore our proposed text generation approach's great potential in deceiving and improving detection models. Our datasets, codes, and annotations are open-sourced.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑