CreBench：从创意、过程到产品的人类对齐创造力评估
CreBench: Human-Aligned Creativity Evaluation from Idea to Process to Product
AI总结：
针对多模态大模型难以对齐人类创造力判断、缺乏相关评估基准的问题，提出CreBench基准与CreMIT数据集，微调得到CreExpert模型，实验证明其创造力评估与人类判断的对齐度优于现有顶尖MLLMs。
摘要：
人类定义的创造力高度抽象，这给多模态大语言模型（MLLMs）理解并评估符合人类判断的创造力带来了挑战，而现有基准的缺失进一步加剧了这一困境。为此，我们提出了CreBench，它包含两个核心组件：1）覆盖从创意构思到创作过程再到最终产品多维度的评估基准；2）CreMIT（创造力多模态指令微调数据集），这是一个多模态创造力评估数据集，包含2.2K条来源多样的多模态数据、79.2K条人类反馈以及4.7M条多类型指令。具体而言，为确保MLLMs能够处理各类与创造力相关的查询，我们提示GPT对这些人类反馈进行优化，以激活更强的创造力评估能力。CreBench为构建能够理解人类对齐创造力的MLLMs提供了基础。基于CreBench，我们对开源通用MLLMs进行微调，得到了多模态创造力评估专家模型CreExpert。大量实验表明，与包括最先进的GPT-4V和Gemini-Pro-Vision在内的当前最优MLLMs相比，我们提出的CreExpert模型在人类创造力评估对齐度上表现显著更优。
英文摘要：
Human-defined creativity is highly abstract, posing a challenge for multimodal large language models (MLLMs) to comprehend and assess creativity that aligns with human judgments. The absence of an existing benchmark further exacerbates this dilemma. To this end, we propose CreBench, which consists of two key components: 1) an evaluation benchmark covering the multiple dimensions from creative idea to process to products; 2) CreMIT (Creativity Multimodal Instruction Tuning dataset), a multimodal creativity evaluation dataset, consisting of 2.2K diverse-sourced multimodal data, 79.2K human feedbacks and 4.7M multi-typed instructions. Specifically, to ensure MLLMs can handle diverse creativity-related queries, we prompt GPT to refine these human feedbacks to activate stronger creativity assessment capabilities. CreBench serves as a foundation for building MLLMs that understand human-aligned creativity. Based on the CreBench, we fine-tune open-source general MLLMs, resulting in CreExpert, a multimodal creativity evaluation expert model. Extensive experiments demonstrate that the proposed CreExpert models achieve significantly better alignment with human creativity evaluation compared to state-of-the-art MLLMs, including the most advanced GPT-4V and Gemini-Pro-Vision.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
