大型语言模型的创造力已达顶峰？LLM间与LLM内变异性分析
Has the Creativity of Large-Language Models peaked? An analysis of inter- and intra-LLM variability
AI总结：
本研究评估14个LLM的创造力，发现其未随时间提升且存在模型内变异性，仅少数响应达人类顶尖水平，强调需更细致评估框架及重视模型选择、提示设计与重复评估。
摘要：
继2023年初ChatGPT被广泛采用后，大量研究报告称大型语言模型（LLMs）在创意任务中能达到甚至超越人类表现。然而，LLMs的创造力是否随时间提升，以及其创意输出的一致性如何，仍不明确。本研究评估了14个广泛使用的LLM——包括GPT-4、Claude、Llama、Grok、Mistral和DeepSeek——通过两项验证过的创造力评估：发散联想任务（DAT）和替代用途任务（AUT）。与预期相反，我们未发现过去18-24个月内创意表现提升的证据，且GPT-4的表现比以往研究更差。在更广泛使用的AUT中，所有模型的平均表现优于人类平均水平，其中GPT-4o和o3-mini表现最佳。但仅有0.28%的LLM生成响应达到人类创造力基准的前10%。除模型间差异外，我们记录到显著的模型内变异性：同一LLM在给定相同提示时，能产生从低于平均水平到原创的输出。这种变异性对创造力研究和实际应用均有重要意义。忽视此类变异性可能导致错误判断LLMs的创意潜力，要么高估要么低估其能力。提示的选择对不同LLM的影响不同。我们的研究结果强调了对更细致的评估框架的需求，并突出了在创意场景中使用生成式AI（GenAI）工具时模型选择、提示设计和重复评估的重要性。
英文摘要：
Following the widespread adoption of ChatGPT in early 2023, numerous studies reported that large language models (LLMs) can match or even surpass human performance in creative tasks. However, it remains unclear whether LLMs have become more creative over time, and how consistent their creative output is. In this study, we evaluated 14 widely used LLMs -- including GPT-4, Claude, Llama, Grok, Mistral, and DeepSeek -- across two validated creativity assessments: the Divergent Association Task (DAT) and the Alternative Uses Task (AUT). Contrary to expectations, we found no evidence of increased creative performance over the past 18-24 months, with GPT-4 performing worse than in previous studies. For the more widely used AUT, all models performed on average better than the average human, with GPT-4o and o3-mini performing best. However, only 0.28% of LLM-generated responses reached the top 10% of human creativity benchmarks. Beyond inter-model differences, we document substantial intra-model variability: the same LLM, given the same prompt, can produce outputs ranging from below-average to original. This variability has important implications for both creativity research and practical applications. Ignoring such variability risks misjudging the creative potential of LLMs, either inflating or underestimating their capabilities. The choice of prompts affected LLMs differently. Our findings underscore the need for more nuanced evaluation frameworks and highlight the importance of model selection, prompt design, and repeated assessment when using Generative AI (GenAI) tools in creative contexts.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
