arXiv 2510.200912026-07-03cs.CLcs.AI

CreativityPrism：大语言模型创造力的跨域评估框架

CreativityPrism: A Cross-Domain Evaluation Framework for Large Language Model Creativity

University of Pittsburgh（匹兹堡大学）；Johns Hopkins University（约翰霍普金斯大学）；University of Notre Dame（诺特丹大学）；University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）；University of Washington（华盛顿大学）；Allen Institute for Artificial Intelligence（人工智能研究院）；University of Massachusetts Amherst（马萨诸塞大学阿默斯特分校）

Zhaoyi Joey Hou, Bowei Alvin Zhang, Yining Lu, Bhiman Kumar Baghel, Anneliese Brei, Ximing Lu, Meng Jiang, Faeze Brahman, Snigdha Chaturvedi, Haw-Shiuan Chang, … 展开作者

Zhaoyi Joey Hou, Bowei Alvin Zhang, Yining Lu, Bhiman Kumar Baghel, Anneliese Brei, Ximing Lu, Meng Jiang, Faeze Brahman, Snigdha Chaturvedi, Haw-Shiuan Chang, Daniel Khashabi, Xiang Lorraine Li

备注：: Published in Transactions on Machine Learning Research (06/2026)

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

提出CreativityPrism框架，整合发散思维、创意写作和逻辑推理三个领域的八项任务，从质量、新颖性和多样性三个维度评估LLM创造力，发现前沿模型在创意写作和逻辑推理上领先，但在发散思维上无显著优势，且各维度间相关性弱。

摘要：

创造力常被视为人类智能的标志。尽管大语言模型（LLM）越来越被认为能生成创造性文本，但仍缺乏跨领域且可扩展的框架来评估其在不同场景下的创造力。现有的LLM创造力评估方法要么严重依赖人类，限制了速度和可扩展性，要么分散在不同领域和不同的创造力定义中。为弥补这一空白，我们提出CreativityPrism，一个评估与分析框架，将来自三个领域（发散思维、创意写作和逻辑推理）的八项任务整合到一个创造力分类体系中，强调LLM生成内容的质量、新颖性和多样性三个维度。该框架设计为可扩展的，并配有经过人工标注验证的可靠自动评估评判器。我们在CreativityPrism上评估了17个最先进的（SoTA）LLM，发现虽然前沿规模的LLM在创意写作和逻辑推理任务上以0.10（或15%）的优势领先于可本地部署的开源模型，但它们在发散思维方面并无显著优势，而发散思维在现有的后训练方案中探索较少。我们的分析还表明，在一个创造性维度或领域中的高表现很少能推广到其他维度或领域；具体来说，新颖性指标通常与其他指标呈现弱相关或负相关。这种碎片化证实了像CreativityPrism这样的跨领域、多维度框架对于任何有意义的LLM创造力评估都是必不可少的。

英文摘要：

Creativity is often seen as a hallmark of human intelligence. While large language models(LLMs) are increasingly perceived as generating creative text, there is still no cross-domain and scalable framework to evaluate their creativity across diverse scenarios. Existing methods of LLM creativity evaluation either heavily rely on humans, limiting speed and scalability, or are fragmented across different domains and different definitions of creativity. To address this gap, we propose CreativityPrism, an evaluation and analysis framework that consolidates eight tasks from three domains: divergent thinking, creative writing, and logical reasoning, into a taxonomy of creativity that emphasizes three dimensions: quality, novelty, and diversity of LLM generations. The framework is designed to be scalable with reliable automatic evaluation judges that have been validated against human annotations. We evaluate 17 state-of-the-art (SoTA) LLMs on CreativityPrism and find that while frontier-scale LLMs dominate creative writing and logical reasoning tasks by a .10 (or 15%) lead over locally-deployable open models, they offer no significant advantage in divergent thinking, a domain much less explored in existing post-training regimes. Our analysis also shows that high performance in one creative dimension or domain rarely generalizes to others; specifically, novelty metrics often show weak or negative correlations with other metrics. This fragmentation confirms that a cross-domain, multi-dimensional framework like CreativityPrism is essential for any meaningful assessment of LLM creativity.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。