arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2304.01487cs.LGcs.AIcs.CL
收藏

用ChatGPT,还是不用ChatGPT:这是一个问题!

To ChatGPT, or not to ChatGPT: That is the question!

Technical University of Darmstadt(达姆施塔特工业大学)

Alessandro Pegoraro, Kavita Kumari, Hossein Fereidooni, Ahmad-Reza Sadeghi

AI总结:

本研究全面评估了最新的ChatGPT检测技术及其他AI文本检测工具,构建了涵盖医疗、问答、金融及社交平台的基准数据集,实验表明现有方法均无法有效检测ChatGPT生成的内容。

摘要:

ChatGPT已成为一种全球现象。随着ChatGPT及其他大型语言模型(LLMs)的出现,以各种方式滥用它们的担忧日益增加,例如传播假新闻、剽窃、操纵公众舆论、作弊和欺诈。因此,区分AI生成与人类生成的内容变得日益重要。研究人员提出了各种检测方法,从基础的二元分类器到更复杂的深度学习模型。一些检测技术依赖于统计特征或句法模式,而另一些则结合语义或上下文信息来提高准确性。本研究的主要目标是对最新的ChatGPT检测技术提供全面且最新的评估。此外,我们还评估了其他未明确声称能检测ChatGPT生成内容的AI生成文本检测工具,以评估它们在检测ChatGPT生成内容方面的表现。为了进行评估,我们整理了一个基准数据集,包含来自ChatGPT和人类的提示,涵盖医疗、开放问答和金融领域的各种问题,以及来自流行社交网络平台的用户生成回复。该数据集作为参考,用于评估各种技术在检测ChatGPT生成内容方面的性能。我们的评估结果表明,现有的方法都无法有效检测ChatGPT生成的内容。

英文摘要:

ChatGPT has become a global sensation. As ChatGPT and other Large Language Models (LLMs) emerge, concerns of misusing them in various ways increase, such as disseminating fake news, plagiarism, manipulating public opinion, cheating, and fraud. Hence, distinguishing AI-generated from human-generated becomes increasingly essential. Researchers have proposed various detection methodologies, ranging from basic binary classifiers to more complex deep-learning models. Some detection techniques rely on statistical characteristics or syntactic patterns, while others incorporate semantic or contextual information to improve accuracy. The primary objective of this study is to provide a comprehensive and contemporary assessment of the most recent techniques in ChatGPT detection. Additionally, we evaluated other AI-generated text detection tools that do not specifically claim to detect ChatGPT-generated content to assess their performance in detecting ChatGPT-generated content. For our evaluation, we have curated a benchmark dataset consisting of prompts from ChatGPT and humans, including diverse questions from medical, open Q&A, and finance domains and user-generated responses from popular social networking platforms. The dataset serves as a reference to assess the performance of various techniques in detecting ChatGPT-generated content. Our evaluation results demonstrate that none of the existing methods can effectively detect ChatGPT-generated content.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑