表征人类与大型语言模型中的创造性过程
Characterising the Creative Process in Humans and Large Language Models
AI总结：
本文提出一种基于句子嵌入和语义相似度的自动化方法，分析人类与LLMs在替代用途任务中的创造性过程，发现LLMs在持续性与灵活性搜索路径上存在任务相关偏向，且更灵活的模型创造力得分更高。
摘要：
大型语言模型（LLMs）表现出相当强的创造性，在创造性任务上常常与普通人类表现相当。然而，关于LLM创造力的研究仅关注“产品”，对创造性“过程”关注甚少。人类创造力的过程分析通常需要人工编码类别或利用反应时间，而这些方法不适用于LLMs。我们提供了一种自动化方法，用于在替代用途任务（Alternate Uses Task）中表征人类和LLMs如何探索语义空间，并与言语流畅性任务（Verbal Fluency Task）中的行为进行对比。我们使用句子嵌入来识别响应类别并计算语义相似度，进而生成跳跃剖面（jump profiles）。我们的结果证实了早期关于人类的研究，报告了通向创造力的两种路径：持续性路径（在少数语义空间中进行深度搜索）和灵活性路径（在多个语义空间中进行广泛搜索），且两种路径都能获得相似的创造力得分。研究发现LLMs偏向于持续性路径或灵活性路径，且这种偏向随任务不同而变化。尽管LLMs作为一个群体与人类剖面相匹配，但它们与创造力之间的关系不同，其中更灵活的模型在创造力上得分更高。我们的数据集和脚本可在GitHub上获取：https://github.com/surabhisnath/Creative_Process。
英文摘要：
Large language models appear quite creative, often performing on par with the average human on creative tasks. However, research on LLM creativity has focused solely on \textit{products}, with little attention on the creative \textit{process}. Process analyses of human creativity often require hand-coded categories or exploit response times, which do not apply to LLMs. We provide an automated method to characterise how humans and LLMs explore semantic spaces on the Alternate Uses Task, and contrast with behaviour in a Verbal Fluency Task. We use sentence embeddings to identify response categories and compute semantic similarities, which we use to generate jump profiles. Our results corroborate earlier work in humans reporting both persistent (deep search in few semantic spaces) and flexible (broad search across multiple semantic spaces) pathways to creativity, where both pathways lead to similar creativity scores. LLMs were found to be biased towards either persistent or flexible paths, that varied across tasks. Though LLMs as a population match human profiles, their relationship with creativity is different, where the more flexible models score higher on creativity. Our dataset and scripts are available on \href{https://github.com/surabhisnath/Creative_Process}{GitHub}.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
