MAiDE-up:GPT生成的酒店评论的多语言欺骗检测
MAiDE-up: Multilingual Deception Detection of GPT-generated Hotel Reviews
AI总结:
针对AI生成虚假酒店评论检测研究多聚焦英语、相关数据集匮乏的问题,本文发布含10种语言各1万条真实与AI生成酒店评论的MAiDE-up数据集,分析多维度因素对检测模型性能的影响。
摘要:
随着大语言模型(LLM)性能提升和普及,欺骗性评论正变得日益常见。尽管现有研究已开发出区分人类真实评论与人类欺骗性评论的模型,但关于真实评论与人工智能撰写的虚假评论之间的区别,人们的了解仍十分有限。此外,目前大多数研究主要聚焦于英语,针对其他语言的研究极少。本文构建并公开发布了MAiDE-up数据集,包含10000条真实酒店评论和10000条人工智能生成的虚假酒店评论,在十种语言间保持均衡。基于该数据集,我们开展了广泛的语言学分析,一是对比人工智能生成的虚假酒店评论与真实酒店评论,二是识别影响欺骗检测模型性能的因素。我们从情感、地域、语言三个主要维度,探究了多种模型在酒店评论欺骗检测任务中的效果,发现这些维度会影响人工智能生成虚假评论的检测准确率。
英文摘要:
Deceptive reviews are becoming increasingly common, especially given the increase in performance and the prevalence of LLMs. While work to date has addressed the development of models to differentiate between truthful and deceptive human reviews, much less is known about the distinction between real reviews and AI-authored fake reviews. Moreover, most of the research so far has focused primarily on English, with very little work dedicated to other languages. In this paper, we compile and make publicly available the MAiDE-up dataset, consisting of 10,000 real and 10,000 AI-generated fake hotel reviews, balanced across ten languages. Using this dataset, we conduct extensive linguistic analyses to (1) compare the AI fake hotel reviews to real hotel reviews, and (2) identify the factors that influence the deception detection model performance. We explore the effectiveness of several models for deception detection in hotel reviews across three main dimensions: sentiment, location, and language. We find that these dimensions influence how well we can detect AI-generated fake reviews.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。