自动诗歌生成中的多样性评估
Evaluating Diversity in Automatic Poetry Generation
AI总结：
本研究评估自动诗歌生成（四行诗）的多样性，比较生成与人类诗歌在结构、词汇、语义和文体上的分布，发现现有系统多样性不足，而文体条件化和字符级建模能显著提升多样性。
摘要：
自然语言生成（NLG），以及更广泛的生成式人工智能，是当前最具影响力的研究领域之一。创意性NLG，如自动诗歌生成，是该领域中一个引人入胜的分支。虽然以往大多数研究在评估自动诗歌生成时侧重于图灵测试的形式——即人类能否区分自动生成与人类创作的诗歌——我们则通过比较生成诗歌与人类诗歌在结构、词汇、语义和文体维度上的分布，来评估自动生成诗歌（重点关注四行诗）的多样性，并评估不同的模型类型（词级与字符级模型、通用大语言模型与诗歌专用模型），包括最新的LLaMA3-8B，以及微调类型（有条件与无条件）。我们发现，当前的自动诗歌系统在多个维度上多样性明显不足——它们往往押韵不够充分，语义上过于统一，甚至无法匹配人类诗歌的长度分布。然而，我们的实验揭示，文体条件化和字符级建模在几乎所有我们探索的维度上显著提高了多样性。我们识别的这些局限性可作为未来更具真正多样性的诗歌生成模型的基础。
英文摘要：
Natural Language Generation (NLG), and more generally generative AI, are among the currently most impactful research fields. Creative NLG, such as automatic poetry generation, is a fascinating niche in this area. While most previous research has focused on forms of the Turing test when evaluating automatic poetry generation -- can humans distinguish between automatic and human generated poetry -- we evaluate the diversity of automatically generated poetry (with a focus on quatrains), by comparing distributions of generated poetry to distributions of human poetry along structural, lexical, semantic and stylistic dimensions, assessing different model types (word vs. character-level, general purpose LLMs vs. poetry-specific models), including the very recent LLaMA3-8B, and types of fine-tuning (conditioned vs. unconditioned). We find that current automatic poetry systems are considerably underdiverse along multiple dimensions -- they often do not rhyme sufficiently, are semantically too uniform and even do not match the length distribution of human poetry. Our experiments reveal, however, that style-conditioning and character-level modeling clearly increases diversity across virtually all dimensions we explore. Our identified limitations may serve as the basis for more genuinely diverse future poetry generation models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
