arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2401.16587cs.CLcs.AIcs.CY
收藏

人类与ChatGPT生成对话的语言学比较

A Linguistic Comparison between Human and ChatGPT-Generated Conversations

Michigan State University(密歇根州立大学);Rochester Institute of Technology(罗切斯特理工学院)

Morgan Sandler, Hyesun Choung, Arun Ross, Prabu David

AI总结:

本研究通过LIWC分析比较人类与ChatGPT生成对话的语言差异,发现ChatGPT在多个语言类别表现更优,并贡献了配套数据集,助力区分人类与LLM文本。

摘要:

本研究探讨了人类与LLM生成对话之间的语言差异,使用由ChatGPT-3.5生成的19.5K对话作为EmpathicDialogues数据集的配套数据。研究采用语言探究与字词统计(LIWC)分析,在118个语言类别上比较了ChatGPT生成的对话与人类对话。结果显示,人类对话具有更大的变异性和真实性,但ChatGPT在社会过程、分析风格、认知、注意力焦点和积极情感基调等类别中表现更优,这印证了近期关于LLM“比人类更像人类”的发现。然而,ChatGPT与人类对话在积极或消极情感方面未发现显著差异。对对话嵌入的分类器分析表明,尽管对话中未明确提及情感,但情感效价被隐式编码。本研究还贡献了一个新颖的配套ChatGPT生成数据集,包含两个独立聊天机器人之间的对话,旨在复制一个广泛用于语言建模AI研究的开放获取人类对话语料库。我们的发现增进了对ChatGPT语言能力的理解,并为区分人类与LLM生成文本的持续努力提供了信息,这对检测AI生成的伪造内容、错误信息和虚假信息至关重要。

英文摘要:

This study explores linguistic differences between human and LLM-generated dialogues, using 19.5K dialogues generated by ChatGPT-3.5 as a companion to the EmpathicDialogues dataset. The research employs Linguistic Inquiry and Word Count (LIWC) analysis, comparing ChatGPT-generated conversations with human conversations across 118 linguistic categories. Results show greater variability and authenticity in human dialogues, but ChatGPT excels in categories such as social processes, analytical style, cognition, attentional focus, and positive emotional tone, reinforcing recent findings of LLMs being "more human than human." However, no significant difference was found in positive or negative affect between ChatGPT and human dialogues. Classifier analysis of dialogue embeddings indicates implicit coding of the valence of affect despite no explicit mention of affect in the conversations. The research also contributes a novel, companion ChatGPT-generated dataset of conversations between two independent chatbots, which were designed to replicate a corpus of human conversations available for open access and used widely in AI research on language modeling. Our findings enhance understanding of ChatGPT's linguistic capabilities and inform ongoing efforts to distinguish between human and LLM-generated text, which is critical in detecting AI-generated fakes, misinformation, and disinformation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑