AGC-Bench: Measuring Artificial General Creativity
Pennsylvania State University(宾夕法尼亚州立大学);University of Massachusetts Lowell(马萨诸塞大学洛厄尔分校);University of Amsterdam(阿姆斯特丹大学);Amazon AGI(亚马逊AGI);Dartmouth College(达特茅斯学院)
Roger Beaty, Vijeta Deshpande, Clin K. Y. Lai, Anna Attuch, Namrata Shivagunde, Swastik Roy, Rajkumar Pujari, Paul V. DiStefano, Sherin Muckatira, Claire E. Ste… 展开作者
Roger Beaty, Vijeta Deshpande, Clin K. Y. Lai, Anna Attuch, Namrata Shivagunde, Swastik Roy, Rajkumar Pujari, Paul V. DiStefano, Sherin Muckatira, Claire E. Stevenson, Mikhail Gronas, Anna Rumshisky
Creativity research has debated whether creativity is domain-specific (e.g., visual, writing, science), and if it is psychometrically separable from general intelligence. Both questions now apply to LLMs, but a unified benchmark of AI creativity remains elusive. We introduce AGC-Bench, an artificial general creativity benchmark built from a systematic review of the AI creativity literature (3,101 papers screened, 497 benchmarks identified), paired with an agentic harness that converts idiosyncratic codebases into HELM-standardized benchmarks. The first release covers 78 datasets spanning brainstorming, problem solving, STEM, narrative, figurative language, and humor. To address bias in LLM-as-judge, we apply Judge Response Theory -- a psychometric calibration of judge leniency/severity; we then fine-tune Qwen3-30B on the bias-corrected ratings of three frontier LLMs to produce AGC-Judge, an open-weight model that robustly scores new creativity benchmarks it was not trained on. Results reveal frontier models at the top of the AGC-Bench leaderboard, with open models close behind. LLMs show different creative strengths, ranking higher on some domains (e.g., writing) than others (e.g., scientific ideation). Extensive experiments yield three main findings. First, applying factor analysis across 83 LLMs, we recover a single creativity factor 'c', analogous to the 'g' factor of general intelligence, that explains 81.5% of variance, related to but separable from general knowledge/reasoning. Second, we show that prompting models to "be creative" boosts their performance far more than enabling reasoning, evidence that the benchmark tracks creativity over general ability. Third, on a human-matched subset, we find the top human still leads the top LLM on creativity. We release AGC-Bench with a public leaderboard, AGC-Judge, and human data as open infrastructure for measuring AI creativity at scale.
CreativeBench: Benchmarking and Enhancing Machine Creativity via Self-Evolving Challenges
Southern University of Science and Technology(南方科技大学);Tsinghua University(清华大学);Peking University(北京大学);University of Bristol(布里斯托大学);The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));Xi’an Jiaotong University(西安交通大学)
Playing with Words, Improving with Rewards: Training Language Models for Creative Association
University of Massachusetts Lowell(马萨诸塞大学洛市分校);Dartmouth College(达特茅斯学院);University of Amsterdam(阿姆斯特丹大学);Pennsylvania State University(宾夕法尼亚州立大学);Amazon AGI(亚马逊人工智能研究院)
Between Sense and Sensibility: Declarative narrativisation of mental models as a basis and benchmark for visuo-spatial cognition and computation focussed collaborative cognitive systems
University of Bremen(不来梅大学)
“\emph{sensing}”和“\emph{sensibility}”之间有什么区别?换句话说,什么样的认知过程介导了感知能力和可感知印象的形成-例如,抽象,类比,假设和理论的形成,信念和他们的修正,论点的形成-在特定领域的问题解决中,还是在日常生活,工作和简单地在环境中走动的常规活动中?如何才能知识和推理的能力,表现出人类在特定的问题背景下,被用来作为一个模型和基准的发展与人类的援助,保证和授权的协作认知(互动)系统?我们提出这些问题的背景下,一系列的辅助技术与\emph{visuo-spatial perception and cognition}任务,包括方面,如常识,创造力,专业领域的知识和解决问题的思维过程的应用。正在考虑的辅助技术包括:(a)人类活动解释;(b)高级认知旋转;(c)建筑和数字媒体创作等领域以人为本的创造性设计;(d)地理信息系统的定性分析。计算叙事不仅提供了丰富的认知基础,但他们也作为一个基准的功能性能在我们的发展计算认知辅助系统。我们认为,有关空间,行动和变化的计算叙事提供了一个有用的模型\emph{visual}和\emph{spatio-temporal thinking}在广泛的解决问题的任务和应用领域,协作认知系统可以提供一个辅助和授权功能。
Divergent Creativity in Humans and Large Language Models
Université de Montréal(蒙特利尔大学);Concordia University(康考迪亚大学);Mila (Quebec AI research Institute)(米拉(魁北克人工智能研究所));University of Toronto Mississauga(多伦多大学密西沙加分校);UNIQUE Center (Quebec Neuro-AI research Center)(UNIQUE中心(魁北克神经人工智能研究中心))
Creativity or Brute Force? Using Brainteasers as a Window into the Problem-Solving Abilities of Large Language Models
Yale University(耶鲁大学);Meta Superintelligence Labs(Meta超智能实验室);Georgia Institute of Technology(佐治亚理工学院);TTIC;Massachusetts Institute of Technology(麻省理工学院)
Evaluating LLMs' Divergent Thinking Capabilities for Scientific Idea Generation with Minimal Context
Gaoling School of Artificial Intelligence, Renmin University of China(中国人民大学人工智能学院);College of Computer Science and Technology, Zhejiang University(浙江大学计算机科学与技术学院);Bank of China(中国银行);School of Engineering Science, University of Chinese Academy of Sciences(中国科学院大学工程科学学院);State Key Laboratory of Nonlinear Mechanics, Institute of Mechanics, Chinese Academy of Sciences(中国科学院力学研究所非线性力学重点实验室)