人类协作是否能提高识别LLM生成的深度伪造文本的准确性？
Does Human Collaboration Enhance the Accuracy of Identifying LLM-Generated Deepfake Texts?
AI总结：
本研究通过AMT和Upwork平台实验，发现人类协作能显著提高检测LLM生成深度伪造文本的准确性，非专家和专家分别提升6.36%和12.76%，并识别出缺乏连贯性为关键指标。
摘要：
大型语言模型（如GPT-4、LLaMA）的进步提高了大规模生成与人类书写相似的连贯句子的能力，从而产生了所谓的深度伪造文本。然而，这一进展带来了安全和隐私方面的担忧，需要有效的解决方案来区分深度伪造文本与人类撰写的文本。尽管先前的研究探讨了人类检测深度伪造文本的能力，但尚未有研究考察人类之间的“协作”是否能提高深度伪造文本的检测。在本研究中，为弥补这一对深度伪造文本理解的空白，我们进行了两组实验：（1）来自AMT平台的非专家个体和（2）来自Upwork平台的写作专家。结果表明，人类之间的协作有可能提高两组对深度伪造文本的检测，非专家和专家的检测准确率分别比个体检测准确率提高了6.36%和12.76%。我们进一步分析了人类用于检测一段文本是否为深度伪造文本的解释，发现深度伪造文本最强烈的指标是其缺乏连贯性和一致性。我们的研究为未来工具和框架设计提供了有用的见解，以促进人类协作检测深度伪造文本。实验数据集和AMT实现可在以下网址获取：https://github.com/huashen218/llm-deepfake-human-study.git
英文摘要：
Advances in Large Language Models (e.g., GPT-4, LLaMA) have improved the generation of coherent sentences resembling human writing on a large scale, resulting in the creation of so-called deepfake texts. However, this progress poses security and privacy concerns, necessitating effective solutions for distinguishing deepfake texts from human-written ones. Although prior works studied humans' ability to detect deepfake texts, none has examined whether "collaboration" among humans improves the detection of deepfake texts. In this study, to address this gap of understanding on deepfake texts, we conducted experiments with two groups: (1) nonexpert individuals from the AMT platform and (2) writing experts from the Upwork platform. The results demonstrate that collaboration among humans can potentially improve the detection of deepfake texts for both groups, increasing detection accuracies by 6.36% for non-experts and 12.76% for experts, respectively, compared to individuals' detection accuracies. We further analyze the explanations that humans used for detecting a piece of text as deepfake text, and find that the strongest indicator of deepfake texts is their lack of coherence and consistency. Our study provides useful insights for future tools and framework designs to facilitate the collaborative human detection of deepfake texts. The experiment datasets and AMT implementations are available at: https://github.com/huashen218/llm-deepfake-human-study.git
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
