提升大语言模型短篇故事生成的多样性
Improving Diversity in LLM Short Story Generation
浏览论文内容
中文总结 AI 辅助
针对LLM生成短篇故事缺乏多样性的问题，提出DivLM后训练框架，通过持续预训练和强化学习提升体裁、语气、风格和命名实体多样性，平均提高多样性指标超9%。
中文摘要 AI 辅助
大型语言模型（LLMs）能够生成准确的回答，但这些回答缺乏多样性。我们尝试针对创意短篇故事生成任务解决这一问题。借鉴既有的写作规范和已知的LLM局限性，我们针对体裁、语气、风格和命名实体等维度进行变化。为了促进这些维度的多样性，我们提出了DivLM，一个包含两个阶段的LLM后训练框架。首先，我们在创意写作语料库上进行持续预训练，并使用权重残差恢复指令跟随能力。然后，我们应用强化学习，采用自定义的复合奖励函数，该函数在保持回答质量的同时，共同最大化目标叙事维度的多样性。我们在两个LLM家族上的实证结果表明，与替代方法相比，DivLM平均将多样性指标提高了9%以上，同时保持了指令跟随、整体回答质量以及与人类输出的相似性。
英文摘要
Large language models (LLMs) can generate accurate responses, but these are void of diversity. We attempt to address this for the task of creative short story generation. Drawing on established writing conventions and known LLM limitations, we target variation in genre, tone, style, and named entities. To promote diversity across these dimensions, we introduce DivLM, an LLM post-training framework consisting of two phases. First, we perform continued pre-training on a creative writing corpus and restore instruction-following capabilities using weight residuals. We then apply reinforcement learning with a custom, composite reward function that jointly maximizes diversity across the targeted narrative dimensions while maintaining response quality. Our empirical results on two LLM families show that DivLM increases diversity metrics by more than 9% on average compared to alternative approaches, while preserving instruction following, overall response quality, and similarity to human outputs.
发表机构
- University College London（伦敦大学学院）
机构由 AI 辅助整理，请以论文原文为准。
