艺术还是巧饰？大语言模型与创造力的虚假承诺
Art or Artifice? Large Language Models and the False Promise of Creativity
AI总结：
针对大语言模型创造力难以客观评估的问题，受TTCT启发提出TTCW测验，经人工评估发现LLM生成故事的创造力远低于专业作品，且LLM作为评估者与专家结果无正相关。
摘要：
研究人员称大语言模型（LLMs）展现出从博客到故事的高质量写作能力，但客观评估一篇作品的创造力颇具挑战。受将创造力作为过程衡量的托兰斯创造性思维测验（TTCT）启发，我们采用共识评估技术[3]，提出托兰斯创意写作测验（TTCW），将创造力作为产品进行评估。TTCW包含14项二元测试，按流畅性、灵活性、独创性、精细性四个原始维度组织。我们招募10名创意写作者，使用TTCW对专业作家或LLMs创作的48篇故事开展人工评估。分析显示，LLM生成的故事通过的TTCW测试数量比专业作家创作的故事少3-10倍。此外，我们探索了用LLM作为评估者自动化TTCW评估的可行性，发现没有任何LLM的评估结果与专家评估呈正相关。
英文摘要：
Researchers have argued that large language models (LLMs) exhibit high-quality writing capabilities from blogs to stories. However, evaluating objectively the creativity of a piece of writing is challenging. Inspired by the Torrance Test of Creative Thinking (TTCT), which measures creativity as a process, we use the Consensual Assessment Technique [3] and propose the Torrance Test of Creative Writing (TTCW) to evaluate creativity as a product. TTCW consists of 14 binary tests organized into the original dimensions of Fluency, Flexibility, Originality, and Elaboration. We recruit 10 creative writers and implement a human assessment of 48 stories written either by professional authors or LLMs using TTCW. Our analysis shows that LLM-generated stories pass 3-10X less TTCW tests than stories written by professionals. In addition, we explore the use of LLMs as assessors to automate the TTCW evaluation, revealing that none of the LLMs positively correlate with the expert assessments.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
