评估大型语言模型与人类语言创造力的一种比较方法
A Comparative Approach to Assessing Linguistic Creativity of Large Language Models and Humans
AI总结：
本文通过包含构词和隐喻任务的通用测试，比较24名人类与24个LLMs的语言创造力，发现LLMs在原创性、精细度和灵活性上整体占优，但人类更倾向扩展型创造力，LLMs更倾向固定型创造力。
摘要：
本文介绍了一种适用于人类和大型语言模型（LLMs）的通用语言创造力测试。该测试由多种任务组成，旨在评估它们基于构词过程（派生和复合）以及隐喻性语言使用来生成新的原创词语和短语的能力。我们对24名人类和同等数量的LLMs进行了测试，并使用OCSAI工具从三个标准自动评估了它们的回答：原创性、精细度和灵活性。结果表明，LLMs不仅在所有评估标准上都优于人类，而且在八项测试任务中的六项上表现更好。随后，我们计算了单个回答的唯一性，结果显示人类与LLMs之间存在一些细微差异。最后，我们对数据集进行了简短的人工分析，发现人类更倾向于E（扩展型）创造力，而LLMs更偏好F（固定型）创造力。
英文摘要：
The following paper introduces a general linguistic creativity test for humans and Large Language Models (LLMs). The test consists of various tasks aimed at assessing their ability to generate new original words and phrases based on word formation processes (derivation and compounding) and on metaphorical language use. We administered the test to 24 humans and to an equal number of LLMs, and we automatically evaluated their answers using OCSAI tool for three criteria: Originality, Elaboration, and Flexibility. The results show that LLMs not only outperformed humans in all the assessed criteria, but did better in six out of the eight test tasks. We then computed the uniqueness of the individual answers, which showed some minor differences between humans and LLMs. Finally, we performed a short manual analysis of the dataset, which revealed that humans are more inclined towards E(extending)-creativity, while LLMs favor F(ixed)-creativity.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
