作为人类的萨列里的AI：通过将机器文本与网络文本进行系统归因来量化语言模型的语言创造力
AI as Humanity's Salieri: Quantifying Linguistic Creativity of Language Models via Systematic Attribution of Machine Text against Web Text
AI总结：
本研究提出量化文本语言创造力的创造力指数及配套动态规划算法DJ SEARCH，发现人类创造力指数高于大语言模型，且该指数可有效用于零样本机器文本检测。
摘要：
长期以来，创造力一直被视为人类智能中最难被AI模仿的方面之一。然而，ChatGPT等大语言模型（Large Language Models，LLMs）的兴起，引发了人们对AI能否匹敌甚至超越人类创造力的质疑。我们提出了创造力指数（CREATIVITY INDEX），作为通过从网络上现有文本片段重构文本来量化文本语言创造力的第一步。创造力指数的提出基于这样一个假设：大语言模型看似卓越的创造力，很大程度上可能源于网络上人类撰写文本的创造力。为了高效计算创造力指数，我们引入了DJ SEARCH，这是一种新型动态规划算法，能够在网络中搜索给定文档中文本片段的逐字匹配和近逐字匹配。实验表明，专业人类作者的创造力指数平均比大语言模型高66.2%，而对齐（alignment）会使大语言模型的创造力指数平均降低30.1%。此外，我们发现海明威等杰出作家的创造力指数显著高于其他人类作者。最后，我们证明创造力指数可作为零样本机器文本检测的极其有效的标准，以30.2%的显著优势超越现有最强的零样本系统DetectGPT，甚至在六个领域中的五个领域里性能超过最强的监督系统GhostBuster。
英文摘要：
Creativity has long been considered one of the most difficult aspect of human intelligence for AI to mimic. However, the rise of Large Language Models (LLMs), like ChatGPT, has raised questions about whether AI can match or even surpass human creativity. We present CREATIVITY INDEX as the first step to quantify the linguistic creativity of a text by reconstructing it from existing text snippets on the web. CREATIVITY INDEX is motivated by the hypothesis that the seemingly remarkable creativity of LLMs may be attributable in large part to the creativity of human-written texts on the web. To compute CREATIVITY INDEX efficiently, we introduce DJ SEARCH, a novel dynamic programming algorithm that can search verbatim and near-verbatim matches of text snippets from a given document against the web. Experiments reveal that the CREATIVITY INDEX of professional human authors is on average 66.2% higher than that of LLMs, and that alignment reduces the CREATIVITY INDEX of LLMs by an average of 30.1%. In addition, we find that distinguished authors like Hemingway exhibit measurably higher CREATIVITY INDEX compared to other human writers. Finally, we demonstrate that CREATIVITY INDEX can be used as a surprisingly effective criterion for zero-shot machine text detection, surpassing the strongest existing zero-shot system, DetectGPT, by a significant margin of 30.2%, and even outperforming the strongest supervised system, GhostBuster, in five out of six domains.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
