arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2305.18149cs.CLcs.AI
收藏

多尺度正样本-无标签(Positive-Unlabeled)AI生成文本检测

Multiscale Positive-Unlabeled Detection of AI-Generated Texts

Peking University(北京大学);Huawei Noah’s Ark Lab(华为诺亚方舟实验室);Huawei Group Finance(华为集团财务);Huawei Central Software Institute(华为中央软件院)

Yuchuan Tian, Hanting Chen, Xutao Wang, Zheyuan Bai, Qinghua Zhang, Ruifeng Li, Chao Xu, Yunhe Wang

AI总结:

针对现有AI文本检测器在短文本上效果不佳的问题,本文提出多尺度正样本-无标签(MPU)训练框架,通过重构PU问题、设计长度敏感损失和文本多尺度模块,同时提升长短文本检测性能,在多个基准上超越现有方法。

摘要:

近期发布的大语言模型(LLM),例如ChatGPT,在生成类人文本方面表现惊人,但可能会影响文本的真实性。已有研究提出了多种检测这类AI生成文本的方法,包括简单的机器学习分类器、基于预训练模型的零样本方法,以及微调后的语言分类模型。然而,主流检测器在短文本(如短信、推文和评论)上的检测效果往往不佳。\n本文提出了一种多尺度正样本-无标签(Multiscale Positive-Unlabeled, MPU)训练框架,在不牺牲长文本检测性能的前提下解决短文本检测的难题。首先,我们注意到短机器生成文本具有类人特性,通过将这些短机器文本视为部分“无标签”样本,将AI文本检测重新定义为部分正样本-无标签(PU)问题。随后,在该PU问题框架下,我们提出了长度敏感的多尺度PU损失(Multiscale PU Loss),其中采用抽象层面的循环模型估计尺度变化语料库的正先验概率。此外,我们引入了文本多尺度化模块来丰富训练语料。\n实验表明,我们的MPU方法提升了长AI生成文本的检测性能,并且显著改善了语言模型检测器的短文本检测效果。采用MPU训练的语言模型在各类短文本和长文本检测基准上均优于现有检测器。代码可在以下链接获取:https://github.com/mindspore-lab/mindone/tree/master/examples/detect_chatgpt 和 https://github.com/YuchuanTian/AIGC_text_detector。

英文摘要:

Recent releases of Large Language Models (LLMs), e.g. ChatGPT, are astonishing at generating human-like texts, but they may impact the authenticity of texts. Previous works proposed methods to detect these AI-generated texts, including simple ML classifiers, pretrained-model-based zero-shot methods, and finetuned language classification models. However, mainstream detectors always fail on short texts, like SMSes, Tweets, and reviews. In this paper, a Multiscale Positive-Unlabeled (MPU) training framework is proposed to address the difficulty of short-text detection without sacrificing long-texts. Firstly, we acknowledge the human-resemblance property of short machine texts, and rephrase AI text detection as a partial Positive-Unlabeled (PU) problem by regarding these short machine texts as partially ``unlabeled". Then in this PU context, we propose the length-sensitive Multiscale PU Loss, where a recurrent model in abstraction is used to estimate positive priors of scale-variant corpora. Additionally, we introduce a Text Multiscaling module to enrich training corpora. Experiments show that our MPU method augments detection performance on long AI-generated texts, and significantly improves short-text detection of language model detectors. Language Models trained with MPU could outcompete existing detectors on various short-text and long-text detection benchmarks. The codes are available at https://github.com/mindspore-lab/mindone/tree/master/examples/detect_chatgpt and https://github.com/YuchuanTian/AIGC_text_detector.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑