Modern AI evaluation frameworks treat evaluator disagreement as noise to be resolved. In creative domains, professional disagreement reflects genuine differences in taste, not measurement error. We argue that evaluating creative AI requires preserving two distinct signals: convergence, where professionals align around shared best practices, and divergence, where individual taste legitimately varies. We present the Human Creativity Benchmark (HCB), a benchmark that operationalizes this separation by collecting pairwise preferences, scalar ratings on prompt adherence, usability, and visual appeal, and qualitative rationale from domain professionals. Across 15,000 professional judgments spanning five creative domains and three workflow phases (ideation, mockup, refinement), we find that convergence concentrates on verifiable dimensions like technical correctness and visual hierarchy, while divergence concentrates on taste-driven dimensions like aesthetic direction and conceptual risk. No model excels uniformly across all phases. Collapsing these signals into a single quality metric discards the most actionable information: where models must be correct versus where they should remain steerable.
CreativeBench: Benchmarking and Enhancing Machine Creativity via Self-Evolving Challenges
Southern University of Science and Technology(南方科技大学);Tsinghua University(清华大学);Peking University(北京大学);University of Bristol(布里斯托大学);The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));Xi’an Jiaotong University(西安交通大学)
Towards creativity characterization of generative models via group-based subset scanning
IBM Research Africa(IBM非洲研究院);IBM Research(IBM研究院)
深度生成模型,如变分自编码器(VAE),已被广泛用于计算创造力的研究。然而,这样的模型不鼓励非分布生成以避免虚假样本生成,限制了它们的创造力。因此,将对人类创造力的研究纳入生成式深度学习技术中,可以使其输出更有吸引力,更像人类。当我们看到面向创造力研究的生成模型的出现时,迫切需要基于机器学习的替代指标来表征这些模型的创造性输出。我们提出了基于组的子集扫描,通过检测生成模型的隐藏层中的异常节点激活的子集来量化,检测和表征创造性过程。我们对原始、典型解码和“创造性解码”(Das et al 2020)图像数据集的实验表明,所提出的子集分数分布对于检测激活空间而不是像素空间中的创造性过程更有用。此外,我们发现,在数据集上,创意样本比正常或非创意样本生成更大的异常子集。在创造性解码过程中突出显示的节点激活与负责正常样本生成的节点激活不同。
Stable diffusion models reveal a persisting human and AI gap in visual creativity
Cognition and Brain Plasticity Unit, Bellvitge Biomedical Research Institute(认知与脑可塑性单位,贝尔维希生物医学研究所);Bridging AI and Neuroscience, Computer Vision Center(弥合人工智能与神经科学,计算机视觉中心);Department of Cognition, Development and Educational Psychology, University of Barcelona(认知、发展与教育心理学系,巴塞罗那大学);Vienna Cognitive Science Hub(维也纳认知科学中心);Faculty of Psychology, University of Vienna(心理学系,维也纳大学);Computer Science Department, Universitat Autonoma de Barcelona(计算机科学系,巴塞罗那自治大学);University of Barcelona Institute for Complex Systems (UBICS)(巴塞罗那大学复杂系统研究所);Department of Catalan Philology and General Linguistics, University of Barcelona(加泰罗尼亚语言学与一般语言学系,巴塞罗那大学);Catalan Institution for Research and Advanced Studies (ICREA)(加泰罗尼亚研究与高级科学研究机构(ICREA));Aix-Marseille University(艾克斯-马赛大学);Institute of Neurosciences (UBNeuro), University of Barcelona(神经科学研究所(UBNeuro),巴塞罗那大学)
Automatic Creativity Measurement in Scratch Programs Across Modalities
Ben-Gurion University of the Negev(内盖夫本-古里安大学);Humboldt University of Berlin(柏林洪堡大学);German Research Center for Artificial Intelligence(德国人工智能研究中心);University of Edinburgh(爱丁堡大学)
Characterising the Creative Process in Humans and Large Language Models
Max Planck Institute for Biological Cybernetics(马克斯·普朗克生物控制论研究所);University of Tübingen(蒂宾根大学);Max Planck School of Cognition(马克斯·普朗克认知科学学院);University of Amsterdam(阿姆斯特丹大学)
National University of Singapore(新加坡国立大学);Stanford University(斯坦福大学);Google DeepMind(谷歌DeepMind);Microsoft Research(微软研究院);Rutgers University(罗格斯大学);University of Pennsylvania(宾夕法尼亚大学);Columbia University(哥伦比亚大学)
CreativityPrism: A Cross-Domain Evaluation Framework for Large Language Model Creativity
University of Pittsburgh(匹兹堡大学);Johns Hopkins University(约翰霍普金斯大学);University of Notre Dame(诺特丹大学);University of North Carolina at Chapel Hill(北卡罗来纳大学教堂山分校);University of Washington(华盛顿大学);Allen Institute for Artificial Intelligence(人工智能研究院);University of Massachusetts Amherst(马萨诸塞大学阿默斯特分校)
Amplifying Human Creativity and Problem Solving with AI Through Generative Collective Intelligence
University of Michigan(密歇根大学);Massachusetts Institute of Technology(麻省理工学院);Stanford Institute for Human-Centered Artificial Intelligence(斯坦福以人为本人工智能研究院);BCG Henderson Institute(波士顿咨询亨德森智库);Xerox Palo Alto Research Center(施乐帕洛阿尔托研究中心)