Pron 对决 Prompt：大型语言模型能否在创意文本写作上挑战世界级小说家？
Pron vs Prompt: Can Large Language Models already Challenge a World-Class Fiction Author at Creative Text Writing?
AI总结：
本文通过让获奖小说家 Patricio Pron 与 GPT-4 进行标题创作和短篇小说写作竞赛，并由文学专家依据 Boden 创造力定义进行人工评估，发现当前 LLMs 仍远未达到顶尖人类创意写作水平，且仅靠扩大模型规模难以实现这一目标。
摘要：
如今，报告大型语言模型（LLMs）在广泛的语言相关任务中超越普通人类的研究结果已成常态，创意文本写作也不例外。因此，提高标准似乎顺理成章：LLMs 是否准备好与顶尖（而非普通）小说家在创意写作技能上一较高下？为了给这一问题提供初步答案，我们举办了一场 Patricio Pron（一位获奖小说家，被认为是他这一代最优秀的作家之一）与 GPT-4（表现最好的 LLMs 之一）之间的竞赛，其精神类似于 DeepBlue 对 Kasparov、AlphaGo 对李世石这样的人机对决。我们要求 Pron 和 GPT-4 各提供三十个标题，然后分别为自己的标题和对手的标题撰写短篇小说。随后，我们受 Boden 对创造力的定义启发，制定了一套评估标准，并收集了由文学评论家和学者提供的 5,400 份人工评估。实验结果表明，LLMs 距离挑战顶尖人类创意作家仍有很大差距，而且达到这种水平的自主创意写作能力很可能无法仅靠更大的语言模型来实现。
英文摘要：
It has become routine to report research results where Large Language Models (LLMs) outperform average humans in a wide range of language-related tasks, and creative text writing is no exception. It seems natural, then, to raise the bid: Are LLMs ready to compete in creative writing skills with a top (rather than average) novelist? To provide an initial answer for this question, we have carried out a contest between Patricio Pron (an awarded novelist, considered one of the best of his generation) and GPT-4 (one of the top performing LLMs), in the spirit of AI-human duels such as DeepBlue vs Kasparov and AlphaGo vs Lee Sidol. We asked Pron and GPT-4 to provide thirty titles each, and then to write short stories for both their titles and their opponent's. Then, we prepared an evaluation rubric inspired by Boden's definition of creativity, and we collected 5,400 manual assessments provided by literature critics and scholars. The results of our experimentation indicate that LLMs are still far from challenging a top human creative writer, and that reaching such level of autonomous creative writing skills probably cannot be reached simply with larger language models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
