评估大语言模型在波斯文学文本生成中的创造力
Evaluating the Creativity of LLMs in Persian Literary Text Generation
AI总结：
本文评估了LLMs在生成波斯文学文本中的创造力，通过定制测试和人工验证，分析其在原创性、流畅性、灵活性和扩展性方面的表现，并探讨其在文学修辞手法应用上的能力。
摘要：
大型语言模型（LLMs）在生成文学文本方面展现了显著的创造力，包括诗歌和短篇故事。然而，先前的研究主要集中在英语上，对非英语文学传统探索有限，且缺乏评估创造力的标准方法。本文评估了LLMs生成富含文化相关表达的波斯文学文本的能力。我们构建了一个涵盖20种不同主题的用户生成波斯文学文本数据集，并通过改编的托兰斯创造性思维测试，在原创性、流畅性、灵活性和扩展性四个维度上评估模型输出。为减少评估成本，我们采用LLM作为评判者进行自动评分，并通过组内相关系数验证其可靠性，观察到高度一致的结果。此外，我们分析了模型在理解和运用四种核心文学修辞手法（明喻、隐喻、夸张和对比）方面的能力。我们的结果突显了LLMs在波斯文学文本生成中的优势和局限性，强调了进一步改进的必要性。
英文摘要：
Large language models (LLMs) have demonstrated notable creative abilities in generating literary texts, including poetry and short stories. However, prior research has primarily centered on English, with limited exploration of non-English literary traditions and without standardized methods for assessing creativity. In this paper, we evaluate the capacity of LLMs to generate Persian literary text enriched with culturally relevant expressions. We build a dataset of user-generated Persian literary spanning 20 diverse topics and assess model outputs along four creativity dimensions-originality, fluency, flexibility, and elaboration-by adapting the Torrance Tests of Creative Thinking. To reduce evaluation costs, we adopt an LLM as a judge for automated scoring and validate its reliability against human judgments using intraclass correlation coefficients, observing strong agreement. In addition, we analyze the models' ability to understand and employ four core literary devices: simile, metaphor, hyperbole, and antithesis. Our results highlight both the strengths and limitations of LLMs in Persian literary text generation, underscoring the need for further refinement.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
