超越复制：一种配对任务框架用于评估LLM在文学翻译中的理解与创造力
Beyond Reproduction: A Paired-Task Framework for Assessing LLM Comprehension and Creativity in Literary Translation
AI总结：
本文提出一种配对任务框架，评估LLM在文学翻译中的理解与创造力。通过11本书的文学片段，评估模型的文本理解与翻译创造力，发现强理解不等于人类水平的创造力，仅Mistral-Large接近人类水平。
摘要：
大型语言模型（LLMs）越来越多地用于创造性任务，如文学翻译。然而，翻译创造力仍被低估，且很少在大规模上进行评估，而源文本理解通常被孤立研究，尽管在专业翻译中，理解和创造力紧密相关。我们通过应用于11本书文学片段的配对任务框架来填补这些空白。任务1评估源文本理解，任务2通过创意潜力单位（UCPs）如隐喻和文字游戏评估翻译创造力。使用结合专家人工注释和基于UCP的自动评分的可扩展评估设置，我们基准测试了23个模型和四个以创造力为导向的提示。我们的发现表明，强理解不等于人类水平的创造力：模型经常产生直译或上下文不合适的翻译，尤其在更远的英语-中文语言对中差距更大。以创造力为导向的提示仅带来小幅提升，仅有一个模型Mistral-Large接近人类水平创造力（0.167 vs. 0.246）。在所有模型-提示组合中，只有三个超过创造力评分0.1，其余仍处于零或接近零水平。
英文摘要：
Large language models (LLMs) are increasingly used for creative tasks such as literary translation. Yet translational creativity remains underexplored and is rarely evaluated at scale, while source-text comprehension is typically studied in isolation, despite the fact that, in professional translation, comprehension and creativity are tightly intertwined. We address these gaps with a paired-task framework applied to literary excerpts from 11 books. Task 1 assesses source-text comprehension, and Task 2 evaluates translational creativity through Units of Creative Potential (UCPs), such as metaphors and wordplay. Using a scalable evaluation setup that combines expert human annotations with UCP-based automatic scoring, we benchmark 23 models and four creativity-oriented prompts. Our findings show that strong comprehension does not translate into human-level creativity: models often produce literal or contextually inappropriate renderings, with particularly large gaps for the more distant English-Chinese language pair. Creativity-oriented prompts yield only modest gains, and only one model, Mistral-Large, comes close to human-level creativity (0.167 vs. 0.246). Across all model-prompt combinations, only three exceed a creativity score of 0.1, while the rest remain at or near zero.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
