arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2401.07867cs.CL
收藏

多语言机器生成文本检测中的作者身份混淆

Authorship Obfuscation in Multilingual Machine-Generated Text Detection

Kempelen Institute of Intelligent Technologies(肯佩伦智能技术研究所);MIT Lincoln Laboratory(麻省理工学院林肯实验室);The Pennsylvania State University(宾夕法尼亚州立大学)

Dominik Macko, Robert Moro, Adaku Uchendu, Ivan Srba, Jason Samuel Lucas, Michiharu Yamashita, Nafis Irtiza Tripto, Dongwon Lee, Jakub Simko, Maria Bielikova

AI总结:

针对多语言机器生成文本(MGT)检测易受作者身份混淆(AO)影响但缺乏多语言环境评估的问题,本研究通过基准测试10种AO方法攻击11种语言下的37种MGT检测方法,填补了这一空白并揭示了AO方法的有效性及部分方法对文本可读性的破坏。

摘要:

近年来大型语言模型(LLMs)的高质量文本生成能力引发了对其滥用的担忧(例如大规模生成/传播虚假信息)。机器生成文本(MGT)检测对于应对此类威胁至关重要,但它易受作者身份混淆(AO)方法的影响,如复述,这会导致MGT逃避检测。迄今为止,这仅在单语环境中进行过评估,因此最近提出的多语言检测器的易感性仍未知。我们通过全面基准测试10种知名AO方法的性能来填补这一空白,针对11种语言的MGT攻击37种MGT检测方法(即10×37×11=4070种组合)。我们还使用混淆文本评估数据增强对 adversarial robustness 的影响。结果表明,所有测试的AO方法都能使所有测试语言中的自动检测失效,其中同形异义攻击特别成功。然而,一些AO方法严重损坏了文本,使其不再可读或难以被人类识别(例如,语言改变、奇怪字符)。

英文摘要:

High-quality text generation capability of recent Large Language Models (LLMs) causes concerns about their misuse (e.g., in massive generation/spread of disinformation). Machine-generated text (MGT) detection is important to cope with such threats. However, it is susceptible to authorship obfuscation (AO) methods, such as paraphrasing, which can cause MGTs to evade detection. So far, this was evaluated only in monolingual settings. Thus, the susceptibility of recently proposed multilingual detectors is still unknown. We fill this gap by comprehensively benchmarking the performance of 10 well-known AO methods, attacking 37 MGT detection methods against MGTs in 11 languages (i.e., 10 $\times$ 37 $\times$ 11 = 4,070 combinations). We also evaluate the effect of data augmentation on adversarial robustness using obfuscated texts. The results indicate that all tested AO methods can cause evasion of automated detection in all tested languages, where homoglyph attacks are especially successful. However, some of the AO methods severely damaged the text, making it no longer readable or easily recognizable by humans (e.g., changed language, weird characters).

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑