小型语言模型在短篇创意写作中可超越人类：一项比较SLM与人类及LLM的研究
Small Language Models can Outperform Humans in Short Creative Writing: A Study Comparing SLMs with Humans and LLMs
AI总结：
本文通过人类评分和语言分析比较微调SLM BART-large、人类及GPT-3.5/GPT-4o的短篇创意写作，发现BART-large总体超越普通人类写作者，证明小模型在特定创意写作场景中可媲美人类与大模型。
摘要：
在本文中，我们评估了经过微调的小型语言模型（SLM）BART-large的创意小说写作能力，并将其表现与人类写作者以及两个大型语言模型（LLM）GPT-3.5和GPT-4o进行比较。我们的评估包含两项实验：（i）一项人类研究，其中68名参与者从语法性、相关性、创造性和吸引力四个维度对人类和SLM创作的短篇故事进行评分；（ii）一项定性语言分析，考察每个模型所生成故事的文本特征。在第一项实验中，BART-large在总体上超过了普通人类写作者（2.11对1.85），相对提升14%，尽管人类在创造性上的微弱优势在统计上并不显著。在第二项实验中，定性分析显示，虽然GPT-4o表现出近乎完美的连贯性并使用了较少的陈词滥调短语，但它倾向于生成更可预测的语言，其故事梗概中只有3%包含令人意外的联想（相比之下BART为15%）。这些发现突出了模型规模和微调如何影响创意写作任务中创造性、流畅性与连贯性之间的平衡，并表明较小的模型在某些情境下能够与人类和更大的模型相抗衡。
英文摘要：
In this paper, we evaluate the creative fiction writing abilities of a fine-tuned small language model (SLM), BART-large, and compare its performance to human writers and two large language models (LLMs): GPT-3.5 and GPT-4o. Our evaluation consists of two experiments: (i) a human study in which 68 participants rated short stories from humans and the SLM on grammaticality, relevance, creativity, and attractiveness, and (ii) a qualitative linguistic analysis examining the textual characteristics of stories produced by each model. In the first experiment, BART-large outscored average human writers overall (2.11 vs. 1.85), a 14% relative improvement, though the slight human advantage in creativity was not statistically significant. In the second experiment, qualitative analysis showed that while GPT-4o demonstrated near-perfect coherence and used less cliche phrases, it tended to produce more predictable language, with only 3% of its synopses featuring surprising associations (compared to 15% for BART). These findings highlight how model size and fine-tuning influence the balance between creativity, fluency, and coherence in creative writing tasks, and demonstrate that smaller models can, in certain contexts, rival both humans and larger models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
