通知：联邦节假日调休：哥伦布日 / 原住民日（Columbus Day / Indigenous Peoples' Day），今日（13日）无更新。通知：联邦节假日调休：哥伦布日 / 原住民日（Columbus Day / Indigenous Peoples' Day），今日（13日）无更新。
arXiv 2606.11762cs.CLcs.AI
语言模型在开放式任务中的自动化创造力评估
Automated Creativity Evaluation of Language Models Across Open-Ended Tasks
Raffles Institution（莱佛士书院）；College of Computing and Data Science, Nanyang Technological University（南洋理工大学计算与数据科学学院）；Lee Kong Chian School of Medicine, Nanyang Technological University（南洋理工大学李光前医学院）；Centre of AI in Medicine (C-AIM), Nanyang Technological University（南洋理工大学人工智能医学中心）
Min Sen Tan, Zachary Kit Chun Choy, Syed Ali Redha Alsagoff, Nadya Yuki Wangsajaya, Mohor Banerjee, Swaagat Bikash Saikia, Alvin Chan
Large language models (LLMs) have achieved remarkable progress in language understanding, reasoning, and generation, sparking growing interest in their creative potential. Realizing this potential requires systematic and scalable methods for evaluating creativity across diverse tasks. However, most existing creativity metrics are tightly coupled to specific tasks, embedding domain assumptions into the evaluation process, and limiting scalability and generality. To address this gap, we introduce an automated, domain-agnostic framework for quantifying LLM creativity across open-ended tasks. Our approach separates the measurement apparatus from the creative task itself, enabling scalable, task-agnostic assessment. Divergent creativity is measured using semantic entropy, a reference-free and robust metric for novelty and diversity, validated against human annotations, LLM-based novelty judgments and baseline diversity measures. Convergent creativity is assessed via a novel retrieval-based multi-agent judge framework that delivers context-sensitive evaluation of task fulfilment with over 60% improved efficiency. We validate our framework in three qualitatively distinct domains: problem-solving (MacGyver), research ideation (HypoGen), and creative writing (BookMIA), using a broad suite of LLMs. Empirical results show that our framework reliably captures key facets of creativity, including novelty, diversity, and task fulfilment, and reveal how model properties, such as size, temperature, recency, and reasoning, impact creative performance. Our work establishes a reproducible and generalizable standard for automated LLM creativity evaluation, paving the way for scalable benchmarking and accelerating progress in creative AI.
CreativityNeuro: Steering Language Model Weights to Improve Divergent Thinking and Reduce Mode Collapse
Center for Brain Science, Harvard University（哈佛大学脑科学中心）；CBS-NTT Program in Physics of Intelligence, Harvard University（哈佛大学CBS-NTT智能物理项目）；Prior Computers；AI Innovation Institute, Stony Brook University（石溪大学人工智能创新研究所）
Decoupled Analysis-Judging: An Automated Creativity Evaluator Using LLMs in Complex Multi-step Creativity Tasks
East China Normal University（华东师范大学）；Shanghai Institute of Artificial Intelligence for Education（上海人工智能教育研究院）；School of Computer Science and Technology, East China Normal University（华东师范大学计算机科学与技术学院）；Yangzhou University（扬州大学）；School of Education and Intelligent Education Research Center, Yangzhou University（扬州大学教育学院与智能教育研究中心）；School of Data Science and Engineering, East China Normal University（华东师范大学数据科学与工程学院）
University of Ottawa（渥太华大学）；American University（美利坚大学）
We apply a large multilingual language model (BLOOM-176B) in open-ended generation of Chinese song lyrics, and evaluate the resulting lyrics for coherence and creativity using human reviewers. We find that current computational metrics for evaluating large language model outputs (MAUVE) have limitations in evaluation of creative writing. We note that the human concept of creativity requires lyrics to be both comprehensible and distinctive -- and that humans assess certain types of machine-generated lyrics to score more highly than real lyrics by popular artists. Inspired by the inherently mult…
AI large language models have (co-)produced amazing written works from newspaper articles to novels and poetry. These works meet the standards of the standard definition of creativity: being original and useful, and sometimes even the additional element of surprise. But can a large language model designed to predict the next text fragment provide creative, out-of-the-box, responses that still solve the problem at hand? We put Open AI's generative natural language model, GPT-3, to the test. Can it provide creative solutions to one of the most commonly used tests in creativity research? We asses…
Characterising the Creative Process in Humans and Large Language Models
Max Planck Institute for Biological Cybernetics（马克斯·普朗克生物控制论研究所）；University of Tübingen（蒂宾根大学）；Max Planck School of Cognition（马克斯·普朗克认知科学学院）；University of Amsterdam（阿姆斯特丹大学）
REC-CBM: Rubric-Aware Error-Correction Concept Bottleneck Models for Trustworthy Open-Ended Grading
School of Computing and Augmented Intelligence, Arizona State University, USA（计算与增强智能学院，亚利桑那州立大学，美国）；Mary Lou Fulton Teachers College, Arizona State University, USA（玛丽·卢·福洛顿教师学院，亚利桑那州立大学，美国）；Department of Computer Science, National Yang Ming Chiao Tung University, TW（国立阳明交通大学计算机科学系，台湾）
A2DEPT: Large Language Model-Driven Automated Algorithm Design via Evolutionary Program Trees
The Institute of Intelligent Computing, University of Electronic Science and Technology of China（电子科技大学智能计算学院）；College of Systems Engineering, National University of Defense Technology（国防科技大学系统工程学院）；Academy of Military Medical Sciences（军事医学科学院）
