Measuring the creativity of large language models (LLMs) is essential for designing methods that can improve creativity and for enhancing our scientific understanding of this ability. To accomplish this, it has become common in recent years to administer tests of human creativity to LLMs. Although these tests provide a convenient and fully automated way to score "creativity," their validity as measures of machine creativity has not been established, and these tests already have limited validity as predictors of human creativity. To address this problem, we conduct the first large-scale, systematic study assessing the effectiveness of human creativity tests for predicting the creative achievement of LLMs across three target constructs: creative writing, divergent thinking, and scientific ideation. We find that the Divergent Association Task (DAT) and the Conditional DAT are the best predictors of creative writing and divergent thinking, respectively, but that test effectiveness varies significantly by construct, and no single test predicts all constructs well. Moreover, contrary to popular belief, no existing test reliably predicts scientific ideation ability. Motivated by this problem, we introduce the Divergent Remote Association Test (DRAT), a vocabulary-space test that assesses both convergent and divergent thinking in a single instrument. The DRAT is the first and only creativity test for LLMs that is a significant predictor of scientific ideation ability, demonstrating robustness across major design choices. Furthermore, the performance gain of the DRAT is not recoverable from any linear combination of the Divergent Association Task and the Remote Associates Test, indicating that assessing divergent and convergent thinking in the same test is essential to reliably predicting scientific ideation ability.
Assessing and Understanding Creativity in Large Language Models
University of Science and Technology of China（中国科学技术大学）；Chinese Academy of Sciences（中国科学院）；Shanghai Innovation Center for Processor Technologies（上海处理器技术创新中心）；University of Chinese Academy of Sciences（中国科学院大学）
Beyond Reproduction: A Paired-Task Framework for Assessing LLM Comprehension and Creativity in Literary Translation
Natural Language Learning Group (NLLG) School of Business Informatics and Mathematics University of Mannheim（曼海姆大学自然语言学习组（NLLG）商学院信息与数学学院）；University of Technology Nuremberg (UTN), Department Engineering（纽伦堡技术大学（UTN）工程系）；University of Gent, Department of Translation, Interpreting and Communication（根特大学翻译、口译与传播系）；University of Aberdeen, Department of Computing Science（阿伯丁大学计算科学系）；Natural Language Learning and Generation (NLLG) Lab（自然语言学习与生成（NLLG）实验室）
CreativityPrism: A Cross-Domain Evaluation Framework for Large Language Model Creativity
University of Pittsburgh（匹兹堡大学）；Johns Hopkins University（约翰霍普金斯大学）；University of Notre Dame（诺特丹大学）；University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）；University of Washington（华盛顿大学）；Allen Institute for Artificial Intelligence（人工智能研究院）；University of Massachusetts Amherst（马萨诸塞大学阿默斯特分校）
Measuring Combinatorial Creativity in Large Language Models and Humans
University of Illinois, Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Siebel School of Computing and Data Science（计算与数据科学学院）；Spiral Works；AI Innovation Institute（人工智能创新研究所）；Stony Brook University（石溪大学）；Simons Institute for the Theory of Computing, Berkeley（伯克利理论计算研究所）
University of Washington（华盛顿大学）；Allen Institute for Artificial Intelligence（艾伦人工智能研究所）；Carnegie Mellon University（卡内基梅隆大学）；University of British Columbia（不列颠哥伦比亚大学）；Vector Institute for AI（矢量人工智能研究所）；Paul G. Allen School of Computer Science & Engineering, University of Washington（华盛顿大学保罗·G·艾伦计算机科学与工程学院）
Leveraging Large Language Models for NLG Evaluation: Advances and Challenges
Peking University（北京大学）；Institute of Information Engineering, CAS（中国科学院信息工程研究所）；AAII, FEIT, University of Technology Sydney（悉尼科技大学）；UCLA（加利福尼亚大学洛杉矶分校）；The Open University of China（国家开放大学）；Beihang University（北京航空航天大学）
Your AI Travel Agent Would Book You a Bullfight: An Agentic Benchmark for Implicit Animal Welfare in Frontier AI Models
Compassion Aligned Machine Learning（同情对齐机器学习）；Sentient Futures（感知未来）；Harvard Kennedy School（哈佛肯尼迪学院）；Appalachian State University Department of Management（阿巴拉契亚州立大学管理系）
ElectriQ: A Benchmark for Assessing the Response Capability of Large Language Models in Power Marketing
Systems Engineering Institute, Xi’an Jiaotong University（系统工程研究所，西安交通大学）；State Key Laboratory of Multiphase Flow in Power Engineering, School of Energy and Power Engineering, Xi’an Jiaotong University（电力工程多相流国家重点实验室，能源与动力工程学院，西安交通大学）；School of Electronic Science and Engineering, Xi'an Jiaotong University（电子科学与工程学院，西安交通大学）；Organizational Management Department, School of Management, Xi’an Jiaotong University（组织管理部，管理学院，西安交通大学）
The Creative Frontier of Generative AI: Managing the Novelty-Usefulness Tradeoff
Samuel Curtis Johnson Graduate School of Management, Cornell University（康奈尔大学塞缪尔·柯蒂斯·约翰逊管理研究生院）；Lee Kong Chian School of Business, Singapore Management University（新加坡管理大学李光前商学院）
Med-V1: Small Language Models for Zero-shot and Scalable Biomedical Evidence Attribution
Division of Intramural Research, National Library of Medicine, National Institutes of Health（国家医学图书馆内部研究部，国立卫生研究院）；Department of Computer Science, University of Virginia（弗吉尼亚大学计算机科学系）；Center for Cancer Research, National Cancer Institute, National Institutes of Health（国家癌症研究所癌症研究中心，国立卫生研究院）；Department of Population Health Sciences, Weill Cornell Medicine Institute of AI for Digital Health, Weill Cornell Medicine（韦尔·科恩医学中心流行病学与健康科学系，韦尔·科恩医学中心人工智能与数字健康研究所）
