arXiv 2405.130122025-07-01cs.CLcs.AI

人类与大型语言模型中的发散性创造力

Divergent Creativity in Humans and Large Language Models

Université de Montréal（蒙特利尔大学）；Concordia University（康考迪亚大学）；Mila (Quebec AI research Institute)（米拉（魁北克人工智能研究所））；University of Toronto Mississauga（多伦多大学密西沙加分校）；UNIQUE Center (Quebec Neuro-AI research Center)（UNIQUE中心（魁北克神经人工智能研究中心））

Antoine Bellemare-Pepin, François Lespinasse, Philipp Thölke, Yann Harel, Kory Mathewson, Jay A. Olson, Yoshua Bengio, Karim Jerbi

备注：: First two and last listed authors are corresponding authors. The first two listed authors contributed equally to this work

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文通过对比10万人类与最先进LLMs的语义发散表现，发现LLMs虽能超越普通人但未达高创造力人类水平，并提出人机基准框架及提示设计、超参数调优等改进方法。

摘要：

近年来大型语言模型（LLMs）的迅猛发展引发了关于其创造力正接近人类水平的论断，这一观点既带来了兴奋也引发了担忧。然而，在这一讨论中一直缺少一个关键环节，即对LLMs的语义多样性进行系统性评估，尤其是与人类发散性思维进行比较。为填补这一空白，我们利用计算创造力领域的最新进展，分析了当前最先进的LLMs以及一个包含100,000名人类受试者的大规模数据集中的语义发散情况。我们发现证据表明，LLMs在发散联想任务（Divergent Association Task）上可以超越普通人的平均表现，并在创造性写作能力上接近人类水平，但仍未达到高创造力人类的典型表现。值得注意的是，即使是表现最好的LLMs，在很大程度上仍被高创造力个体超越，这凸显了当前LLMs尚无法突破的上限。我们提出的人机基准测试框架回应了关于AI即将取代人类创造性劳动的争论，利用既定的客观度量标准来区分各自创造性语言输出的质量。在推动对人类创造性思维与AI系统相比所具有的独特要素进行更深入探索的同时，我们提出了一系列技术手段，例如提示设计和超参数调优，以改善模型输出在语义多样性方面的表现。

英文摘要：

The recent surge of Large Language Models (LLMs) has led to claims that they are approaching a level of creativity akin to human capabilities. This idea has sparked a blend of excitement and apprehension. However, a critical piece that has been missing in this discourse is a systematic evaluation of LLMs' semantic diversity, particularly in comparison to human divergent thinking. To bridge this gap, we leverage recent advances in computational creativity to analyze semantic divergence in both state-of-the-art LLMs and a substantial dataset of 100,000 humans. We found evidence that LLMs can surpass average human performance on the Divergent Association Task, and approach human creative writing abilities, though they fall short of the typical performance of highly creative humans. Notably, even the top performing LLMs are still largely surpassed by highly creative individuals, underscoring a ceiling that current LLMs still fail to surpass. Our human-machine benchmarking framework addresses the polemic surrounding the imminent replacement of human creative labour by AI, disentangling the quality of the respective creative linguistic outputs using established objective measures. While prompting deeper exploration of the distinctive elements of human inventive thought compared to those of AI systems, we lay out a series of techniques to improve their outputs with respect to semantic diversity, such as prompt design and hyper-parameter tuning.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。