探究大型语言模型在Reddit Showerthoughts领域特定写作风格适配中的机智性、创造力与可检测性
Investigating Wit, Creativity, and Detectability of Large Language Models in Domain-Specific Writing Style Adaptation of Reddit's Showerthoughts
AI总结：
本文研究不同规模LLM在Reddit Showerthoughts创意短文本中模仿人类风格的能力，发现人类评估者虽认为生成文本创意质量略低但无法可靠识别AI生成内容，并发布了相关数据集。
摘要：
近期的大型语言模型（LLMs）已展现出生成难以或无法与人类写作区分的内容的能力。我们研究了不同规模的大型语言模型在Showerthoughts（即人们在日常琐碎活动中可能产生的想法）这一短篇创意文本领域中复制人类写作风格的能力。我们将基于Reddit数据微调的GPT-2和GPT-Neo，以及以零样本方式调用的GPT-3.5，与人类撰写的文本进行了比较。我们测量了人类对文本在衡量创意、机智文本质量的具体维度上的偏好。此外，我们比较了人类与微调的RoBERTa分类器检测AI生成文本的能力。我们得出结论：人类评估者对生成文本的创意质量平均评分略低，但他们无法可靠地区分人类撰写的文本与AI生成的文本。我们还基于Reddit Showerthoughts帖子提供了一个用于创意、机智文本生成的数据集。
英文摘要：
Recent Large Language Models (LLMs) have shown the ability to generate content that is difficult or impossible to distinguish from human writing. We investigate the ability of differently-sized LLMs to replicate human writing style in short, creative texts in the domain of Showerthoughts, thoughts that may occur during mundane activities. We compare GPT-2 and GPT-Neo fine-tuned on Reddit data as well as GPT-3.5 invoked in a zero-shot manner, against human-authored texts. We measure human preference on the texts across the specific dimensions that account for the quality of creative, witty texts. Additionally, we compare the ability of humans versus fine-tuned RoBERTa classifiers to detect AI-generated texts. We conclude that human evaluators rate the generated texts slightly worse on average regarding their creative quality, but they are unable to reliably distinguish between human-written and AI-generated texts. We further provide a dataset for creative, witty text generation based on Reddit Showerthoughts posts.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
