arXiv 2402.011582024-02-02cs.CL

LLM-Detector：通过开源大语言模型指令微调提升AI生成中文文本检测效果

LLM-Detector: Improving AI-Generated Chinese Text Detection with Open-Source LLM Instruction Tuning

Macao Polytechnic University（澳门理工大学）；Fudan University（复旦大学）；Wuhan University（武汉大学）

Rongsheng Wang, Haoming Chen, Ruizhe Zhou, Han Ma, Yaofei Duan, Yanlan Kang, Songhua Yang, Baoyu Fan, Tao Tan

备注：: 17 pages, 13 tables, 7 figures

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对现有AI生成中文文本检测模型域内过拟合、域外及句子级检测效果差的问题，提出基于开源LLM指令微调的LLM-Detector，在两级检测任务上均优于基线且泛化性强、易部署。

摘要：

ChatGPT及其他通用大语言模型（LLM）已取得显著成效，但也引发了人们对AI生成文本被滥用的担忧。现有基于BERT、RoBERTa等架构的AI生成文本检测模型易出现域内过拟合问题，导致域外（OOD）检测性能不佳。本文首先收集了人类专家与9种LLM针对多领域问题生成的中文文本回复，进一步构建了混合人类撰写句子与LLM润色句子的数据集。随后我们提出LLM-Detector，这是一种通过对LLM进行指令微调、同时实现文档级与句子级文本检测的新方法。该方法利用LLM在预训练阶段习得的丰富知识，使其能够检测自身生成的文本；指令微调则让模型的输出与用户预期的文本检测任务对齐。实验结果表明，现有方法在句子级AI生成文本检测与域外检测任务中表现不佳；相比之下，本文提出的方法不仅在句子级与文档级文本检测任务上均显著优于基线方法，还展现出强大的泛化能力。此外，由于LLM-Detector基于开源LLM训练，易于定制化部署。

英文摘要：

ChatGPT and other general large language models (LLMs) have achieved remarkable success, but they have also raised concerns about the misuse of AI-generated texts. Existing AI-generated text detection models, such as based on BERT and RoBERTa, are prone to in-domain over-fitting, leading to poor out-of-domain (OOD) detection performance. In this paper, we first collected Chinese text responses generated by human experts and 9 types of LLMs, for which to multiple domains questions, and further created a dataset that mixed human-written sentences and sentences polished by LLMs. We then proposed LLM-Detector, a novel method for both document-level and sentence-level text detection through Instruction Tuning of LLMs. Our method leverages the wealth of knowledge LLMs acquire during pre-training, enabling them to detect the text they generate. Instruction tuning aligns the model's responses with the user's expected text detection tasks. Experimental results show that previous methods struggle with sentence-level AI-generated text detection and OOD detection. In contrast, our proposed method not only significantly outperforms baseline methods in both sentence-level and document-level text detection but also demonstrates strong generalization capabilities. Furthermore, since LLM-Detector is trained based on open-source LLMs, it is easy to customize for deployment.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。