arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2402.14533cs.CL
收藏

这到底是谁的大语言模型?针对GPT-3.5、GPT-4与Bard的语言学比较与大语言模型归因研究

Whose LLM is it Anyway? Linguistic Comparison and LLM Attribution for GPT-3.5, GPT-4 and Bard

Department of Information Science(信息科学系);Bar Ilan University(巴伊兰大学);Department of Information Systems(信息系统系);University of Haifa(海法大学);Department of Computing(计算机系);Jonkoping University(约翰·科普林大学)

Ariel Rosenfeld, Teddy Lazebnik

AI总结:

该研究通过语言学分析对比GPT-3.5、GPT-4、Bard的文本语言特征,发现三者存在显著差异,可借助现成分类模型以88%的准确率实现文本的大语言模型来源归因。

摘要:

大语言模型(LLM)生成的文本质量可与人类水平相当甚至超越人类,但目前尚不清楚LLM是否会像人类作者一样展现出独特的语言风格。我们通过全面的语言学分析,对比了当前最流行的三款大语言模型(GPT-3.5、GPT-4和Bard)针对多样化输入生成文本的词汇、词性(POS)分布、依存关系分布及情感特征。结果显示三者存在显著的语言差异,借此我们使用简单的现成分类模型,即可将给定文本归因至其生成的大语言模型来源,准确率达到可观的88%。本文还讨论了这一有趣发现的理论与实践意义。

英文摘要:

Large Language Models (LLMs) are capable of generating text that is similar to or surpasses human quality. However, it is unclear whether LLMs tend to exhibit distinctive linguistic styles akin to how human authors do. Through a comprehensive linguistic analysis, we compare the vocabulary, Part-Of-Speech (POS) distribution, dependency distribution, and sentiment of texts generated by three of the most popular LLMS today (GPT-3.5, GPT-4, and Bard) to diverse inputs. The results point to significant linguistic variations which, in turn, enable us to attribute a given text to its LLM origin with a favorable 88\% accuracy using a simple off-the-shelf classification model. Theoretical and practical implications of this intriguing finding are discussed.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑