arXiv 2504.131202025-04-29cs.CVcs.AIcs.CL

探究与诱导视觉语言模型的组合创造力

Probing and Inducing Combinational Creativity in Vision-Language Models

Institute for Artificial Intelligence, Peking University（北京大学人工智能研究院）；Yuanpei College, Peking University（北京大学元培学院）；State Key Laboratory of General Artificial Intelligence, BIGAI（人工智能通用基础理论国家重点实验室）；Center on Frontiers of Computing Studies, School of Computer Science, Peking University（计算机科学学院前沿计算研究中心）

Yongqian Peng, Yuxi Ma, Mengmeng Wang, Yuxuan Wang, Yizhou Wang, Chi Zhang, Yixin Zhu, Zilong Zheng

备注：: Project page: https://ppyyqq.github.io/aicc/ The first two authors contribute equally

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

提出IEI框架从概念融合视角探究视觉语言模型的组合创造力，并构建CreativeMashup数据集验证其有效性，发现模型在理解任务超越普通人类但未达专家水平，在生成任务中能显著提升输出创造性。

摘要：

将现有概念组合成新颖想法的能力是人类智能的基本标志。GPT-4V和DALLE-3等视觉语言模型（VLMs）的最新进展引发了争论：其输出是否反映了组合创造力——由M. A. Boden（1998）定义为通过组合现有概念来综合新颖想法——还是对训练数据的复杂模式匹配。受认知科学启发，我们从概念融合的视角研究VLMs的组合创造力。我们提出了Identification-Explanation-Implication（IEI）框架，将创作过程分解为三个层次：识别输入空间、提取共享属性和推导新颖的语义蕴含。为验证该框架，我们整理了CreativeMashup，一个包含666个艺术家生成的视觉混搭的高质量数据集，并根据IEI框架进行了标注。通过大量实验，我们证明在理解任务中，最好的VLMs已超越普通人类表现，但尚未达到专家级理解水平；在生成任务中，将我们的IEI框架纳入生成流程显著提升了VLMs输出的创造性质量。我们的发现为评估人工创造力奠定了理论基础，并为改进VLMs的创造性生成提供了实用指南。

英文摘要：

The ability to combine existing concepts into novel ideas stands as a fundamental hallmark of human intelligence. Recent advances in Vision-Language Models (VLMs) like GPT-4V and DALLE-3 have sparked debate about whether their outputs reflect combinational creativity--defined by M. A. Boden (1998) as synthesizing novel ideas through combining existing concepts--or sophisticated pattern matching of training data. Drawing inspiration from cognitive science, we investigate the combinational creativity of VLMs from the lens of concept blending. We propose the Identification-Explanation-Implication (IEI) framework, which decomposes creative processes into three levels: identifying input spaces, extracting shared attributes, and deriving novel semantic implications. To validate this framework, we curate CreativeMashup, a high-quality dataset of 666 artist-generated visual mashups annotated according to the IEI framework. Through extensive experiments, we demonstrate that in comprehension tasks, best VLMs have surpassed average human performance while falling short of expert-level understanding; in generation tasks, incorporating our IEI framework into the generation pipeline significantly enhances the creative quality of VLMs' outputs. Our findings establish both a theoretical foundation for evaluating artificial creativity and practical guidelines for improving creative generation in VLMs.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。