Putting GPT-3's Creativity to the (Alternative Uses) Test
University of Amsterdam（阿姆斯特丹大学）
Claire Stevenson, Iris Smal, Matthijs Baas, Raoul Grasman, Han van der Maas
备注：
5 pages, 6 figures, accepted at the International Conference on Computational Creativity (ICCC) 2022 as a Short Paper. See https://osf.io/vmk3c/ for data, analyses and code
AI large language models have (co-)produced amazing written works from newspaper articles to novels and poetry. These works meet the standards of the standard definition of creativity: being original and useful, and sometimes even the additional element of surprise. But can a large language model designed to predict the next text fragment provide creative, out-of-the-box, responses that still solve the problem at hand? We put Open AI's generative natural language model, GPT-3, to the test. Can it provide creative solutions to one of the most commonly used tests in creativity research? We assessed GPT-3's creativity on Guilford's Alternative Uses Test and compared its performance to previously collected human responses on expert ratings of originality, usefulness and surprise of responses, flexibility of each set of ideas as well as an automated method to measure creativity based on the semantic distance between a response and the AUT object in question. Our results show that -- on the whole -- humans currently outperform GPT-3 when it comes to creative output. But, we believe it is only a matter of time before GPT-3 catches up on this particular task. We discuss what this work reveals about human and AI creativity, creativity testing and our definition of creativity.
Putting GPT-4o to the Sword: A Comprehensive Evaluation of Language, Vision, Speech, and Multimodal Proficiency
University of Guelph（圭尔夫大学）；University of North Texas（北德克萨斯大学）；Iowa State University（爱荷华州立大学）；Zayed University（扎耶德大学）；University of Texas at Dallas（德克萨斯大学达拉斯分校）；National University of Computer & Emerging Sciences（国立计算机与新兴科学大学）
Stable diffusion models reveal a persisting human and AI gap in visual creativity
Cognition and Brain Plasticity Unit, Bellvitge Biomedical Research Institute（认知与脑可塑性单位，贝尔维希生物医学研究所）；Bridging AI and Neuroscience, Computer Vision Center（弥合人工智能与神经科学，计算机视觉中心）；Department of Cognition, Development and Educational Psychology, University of Barcelona（认知、发展与教育心理学系，巴塞罗那大学）；Vienna Cognitive Science Hub（维也纳认知科学中心）；Faculty of Psychology, University of Vienna（心理学系，维也纳大学）；Computer Science Department, Universitat Autonoma de Barcelona（计算机科学系，巴塞罗那自治大学）；University of Barcelona Institute for Complex Systems (UBICS)（巴塞罗那大学复杂系统研究所）；Department of Catalan Philology and General Linguistics, University of Barcelona（加泰罗尼亚语言学与一般语言学系，巴塞罗那大学）；Catalan Institution for Research and Advanced Studies (ICREA)（加泰罗尼亚研究与高级科学研究机构（ICREA））；Aix-Marseille University（艾克斯-马赛大学）；Institute of Neurosciences (UBNeuro), University of Barcelona（神经科学研究所（UBNeuro），巴塞罗那大学）
Probing and Inducing Combinational Creativity in Vision-Language Models
Institute for Artificial Intelligence, Peking University（北京大学人工智能研究院）；Yuanpei College, Peking University（北京大学元培学院）；State Key Laboratory of General Artificial Intelligence, BIGAI（人工智能通用基础理论国家重点实验室）；Center on Frontiers of Computing Studies, School of Computer Science, Peking University（计算机科学学院前沿计算研究中心）
Poetry Generation involves teaching systems to automatically generate text that resembles poetic work. A deep learning system can learn to generate poetry on its own by training on a corpus of poems and modeling the particular style of language. In this paper, we propose taking an approach that fine-tunes GPT-2, a pre-trained language model, to our downstream task of poetry generation. We extend prior work on poetry generation by introducing creative elements. Specifically, we generate poems that express emotion and elicit the same in readers, and poems that use the language of dreams---called…
Creativity is a deeply debated topic, as this concept is arguably quintessential to our humanity. Across different epochs, it has been infused with an extensive variety of meanings relevant to that era. Along these, the evolution of technology have provided a plurality of novel tools for creative purposes. Recently, the advent of Artificial Intelligence (AI), through deep learning approaches, have seen proficient successes across various applications. The use of such technologies for creativity appear in a natural continuity to the artistic trend of this century. However, the aura of a technol…
GAUSS: Benchmarking Structured Mathematical Skills for Large Language Models
Hyperbolic Labs；California Institute of Technology（加州理工学院）；University of California, Berkeley（加州大学伯克利分校）；Stanford University（斯坦福大学）；Nvidia（英伟达）；University of Washington（华盛顿大学）；University of Hong Kong（香港大学）
Self-Instruct: Aligning Language Models with Self-Generated Instructions
University of Washington（华盛顿大学）；Tehran Polytechnic（德黑兰理工大学）；Arizona State University（亚利桑那州立大学）；Johns Hopkins University（约翰斯·霍普金斯大学）；Allen Institute for AI（艾伦人工智能研究所）
Pron vs Prompt: Can Large Language Models already Challenge a World-Class Fiction Author at Creative Text Writing?
School of Computer Science, UNED, Madrid, Spain（计算机科学学院，UNED，马德里，西班牙）；Faculty of Education, UCM, Madrid, Spain（教育学院，UCM，马德里，西班牙）；Faculty of Philosophy, UNED, Madrid, Spain（哲学学院，UNED，马德里，西班牙）
Evaluating Morphological Compositional Generalization in Large Language Models
EPFL（瑞士联邦理工学院）；Idiap Research Institute（Idiap研究机构）；Duke University（杜克大学）；Brandeis University（布兰迪斯大学）；LMU Munich（慕尼黑大学）；University of Cambridge（剑桥大学）；Johns Hopkins University（约翰霍普金斯大学）；Università della Svizzera Italiana（瑞士意大利大学）；New York University（纽约大学）
Automatic Item Generation for Personality Situational Judgment Tests with Large Language Models
Beijing Normal University（北京师范大学）；Central China Normal University（华中师范大学）；Key Laboratory of Adolescent Cyberpsychology and Behavior, Ministry of Education（教育部青少年网络心理与行为重点实验室）；Beijing Key Laboratory of Applied Experimental Psychology（北京市应用实验心理学重点实验室）；Research Center for Capacity Building in Educational Assessment and Evaluation (Beijing Higher Education Innovation Center for Philosophy and Social Sciences)（教育评价与评估能力建设研究中心（北京高等教育哲学社会科学创新中心））
This project aims to produce the next volume of machine-generated poetry, a complex art form that can be structured and unstructured, and carries depth in the meaning between the lines. GPoeT-2 is based on fine-tuning a state of the art natural language model (i.e. GPT-2) to generate limericks, typically humorous structured poems consisting of five lines with a AABBA rhyming scheme. With a two-stage generation system utilizing both forward and reverse language modeling, GPoeT-2 is capable of freely generating limericks in diverse topics while following the rhyming structure without any seed ph…
Dialogue with the Machine and Dialogue with the Art World: Evaluating Generative AI for Culturally-Situated Creativity
Google Research（谷歌研究院）；Google DeepMind（谷歌DeepMind）；USC School of Architecture（南加州大学建筑学院）；Farbod Mehr Studio（法博德·梅尔工作室）；MIT（麻省理工学院）；Cornell University（康奈尔大学）
Before and After Temperature: A Distributional View of Creative LLM Generation
University of California, San Diego（加州大学圣地亚哥分校）；Indian Institute of Technology, Kharagpur（印度理工学院克哈格布尔分校）；Delhi Technological University（德里技术大学）；Carnegie Mellon University（卡内基梅隆大学）；Rutgers University（罗格斯大学）；Georgia Institute of Technology（佐治亚理工学院）
A Novel Mathematical Framework for Objective Characterization of Ideas
Department of Mechanical Engineering, Indian Institute of Science (IISc), Bangalore, India - 560012（机械工程系，印度科学研究院（IISc），班加罗尔，印度 - 560012）；Department of Design and Manufacturing (erstwhile CPDM), Indian Institute of Science (IISc), Bangalore, India - 560012（设计与制造系（原CPDM），印度科学研究院（IISc），班加罗尔，印度 - 560012）
本文提出一种自动化数学框架，将想法表示为高维向量并结合 UMAP、DBSCAN 和 PCA 量化多样性，以客观筛选 CAI 或人类生成的创意。
SORRY-Bench: Systematically Evaluating Large Language Model Safety Refusal
Princeton University（普林斯顿大学）；Virginia Tech（弗吉尼亚理工大学）；Stanford University（斯坦福大学）；UC Berkeley（加州大学伯克利分校）；University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；University of Chicago（芝加哥大学）
Investigating the performance of Retrieval-Augmented Generation and fine-tuning for the development of AI-driven knowledge-based systems
Department of Data Science and Visualization, Faculty of Informatics, University of Debrecen（数据科学与可视化系，信息学院，德布勒恩大学）；Doctoral School of Informatics, University of Debrecen（信息学博士学校，德布勒恩大学）；Neumann Technology Platform, Neumann Nonprofit Ltd.（内梅尔技术平台，内梅尔非营利有限公司）；Data-Driven Health Division of National Laboratory for Health Security, Health Services Management Training Centre, Semmelweis University（国家安全国家实验室数据驱动健康部，卫生服务管理培训中心，塞梅尔维斯大学）
