We introduce Creativity Benchmark, an evaluation framework for large language models (LLMs) in marketing creativity. The benchmark covers 100 brands (12 categories) and three prompt types (Insights, Ideas, Wild Ideas). Human pairwise preferences from 678 practising creatives over 11,012 anonymised comparisons, analysed with Bradley-Terry models, show tightly clustered performance with no model dominating across brands or prompt types: the top-bottom spread is $Δθ\approx 0.45$, which implies a head-to-head win probability of $0.61$; the highest-rated model beats the lowest only about $61\%$ of the time. We also analyse model diversity using cosine distances to capture intra- and inter-model variation and sensitivity to prompt reframing. Comparing three LLM-as-judge setups with human rankings reveals weak, inconsistent correlations and judge-specific biases, underscoring that automated judges cannot substitute for human evaluation. Conventional creativity tests also transfer only partially to brand-constrained tasks. Overall, the results highlight the need for expert human evaluation and diversity-aware workflows.
MindFuse: Towards GenAI Explainability in Marketing Strategy Co-Creation
ITMO University(ITMO大学);SoMin.ai Research(SoMin.ai研究);Faculty of Software Engineering(软件工程学院);Computer Systems, ITMO University(计算机系统,ITMO大学);ISP RAS Research Center for Trusted Artificial Intelligence(俄罗斯科学院信息与系统研究所以及可信人工智能研究中心)
ElectriQ: A Benchmark for Assessing the Response Capability of Large Language Models in Power Marketing
Systems Engineering Institute, Xi’an Jiaotong University(系统工程研究所,西安交通大学);State Key Laboratory of Multiphase Flow in Power Engineering, School of Energy and Power Engineering, Xi’an Jiaotong University(电力工程多相流国家重点实验室,能源与动力工程学院,西安交通大学);School of Electronic Science and Engineering, Xi'an Jiaotong University(电子科学与工程学院,西安交通大学);Organizational Management Department, School of Management, Xi’an Jiaotong University(组织管理部,管理学院,西安交通大学)
Beyond Reproduction: A Paired-Task Framework for Assessing LLM Comprehension and Creativity in Literary Translation
Natural Language Learning Group (NLLG) School of Business Informatics and Mathematics University of Mannheim(曼海姆大学自然语言学习组(NLLG)商学院信息与数学学院);University of Technology Nuremberg (UTN), Department Engineering(纽伦堡技术大学(UTN)工程系);University of Gent, Department of Translation, Interpreting and Communication(根特大学翻译、口译与传播系);University of Aberdeen, Department of Computing Science(阿伯丁大学计算科学系);Natural Language Learning and Generation (NLLG) Lab(自然语言学习与生成(NLLG)实验室)
Playing with Words, Improving with Rewards: Training Language Models for Creative Association
University of Massachusetts Lowell(马萨诸塞大学洛市分校);Dartmouth College(达特茅斯学院);University of Amsterdam(阿姆斯特丹大学);Pennsylvania State University(宾夕法尼亚州立大学);Amazon AGI(亚马逊人工智能研究院)
Automated Creativity Evaluation of Language Models Across Open-Ended Tasks
Raffles Institution(莱佛士书院);College of Computing and Data Science, Nanyang Technological University(南洋理工大学计算与数据科学学院);Lee Kong Chian School of Medicine, Nanyang Technological University(南洋理工大学李光前医学院);Centre of AI in Medicine (C-AIM), Nanyang Technological University(南洋理工大学人工智能医学中心)
Divergent Creativity in Humans and Large Language Models
Université de Montréal(蒙特利尔大学);Concordia University(康考迪亚大学);Mila (Quebec AI research Institute)(米拉(魁北克人工智能研究所));University of Toronto Mississauga(多伦多大学密西沙加分校);UNIQUE Center (Quebec Neuro-AI research Center)(UNIQUE中心(魁北克神经人工智能研究中心))
Creativity or Brute Force? Using Brainteasers as a Window into the Problem-Solving Abilities of Large Language Models
Yale University(耶鲁大学);Meta Superintelligence Labs(Meta超智能实验室);Georgia Institute of Technology(佐治亚理工学院);TTIC;Massachusetts Institute of Technology(麻省理工学院)
Using Large Language Models to Create AI Personas for Replication, Generalization and Prediction of Media Effects: An Empirical Test of 133 Published Experimental Research Findings
BILLY: Steering Large Language Models via Merging Persona Vectors for Creative Generation
Department of Electrical Engineering, National Taiwan University(国立台湾大学电子工程系);Department of Computer Science & Information Engineering, National Taiwan University(国立台湾大学计算机科学与信息工程系);Graduate Institute of Communication Engineering, National Taiwan University(国立台湾大学通信工程研究所);CSAIL, Massachusetts Institute of Technology(麻省理工学院计算机科学与人工智能实验室)
Measuring Combinatorial Creativity in Large Language Models and Humans
University of Illinois, Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Siebel School of Computing and Data Science(计算与数据科学学院);Spiral Works;AI Innovation Institute(人工智能创新研究所);Stony Brook University(石溪大学);Simons Institute for the Theory of Computing, Berkeley(伯克利理论计算研究所)
University of Washington(华盛顿大学);Allen Institute for Artificial Intelligence(艾伦人工智能研究所);Carnegie Mellon University(卡内基梅隆大学);University of British Columbia(不列颠哥伦比亚大学);Vector Institute for AI(矢量人工智能研究所);Paul G. Allen School of Computer Science & Engineering, University of Washington(华盛顿大学保罗·G·艾伦计算机科学与工程学院)
GAUSS: Benchmarking Structured Mathematical Skills for Large Language Models
Hyperbolic Labs;California Institute of Technology(加州理工学院);University of California, Berkeley(加州大学伯克利分校);Stanford University(斯坦福大学);Nvidia(英伟达);University of Washington(华盛顿大学);University of Hong Kong(香港大学)
CreativeBench: Benchmarking and Enhancing Machine Creativity via Self-Evolving Challenges
Southern University of Science and Technology(南方科技大学);Tsinghua University(清华大学);Peking University(北京大学);University of Bristol(布里斯托大学);The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));Xi’an Jiaotong University(西安交通大学)