深度关联，高度创造力：一种简单而有效的大语言模型评估指标
Deep Associations, High Creativity: A Simple yet Effective Metric for Evaluating Large Language Models
AI总结：
本文提出PACE指标，通过让大语言模型生成平行关联链来评估创造力，该方法抗数据污染且高效，与人类评估高度相关，并发现专业人类在联想创造力上仍优于高性能模型。
摘要：
大语言模型（LLM）创造力的评估是一个关键的研究领域，尽管数据污染和昂贵的人工评估等挑战常常阻碍进展。受人类创造力评估的启发，我们提出了PACE方法，要求LLM生成平行关联链以评估其创造力。PACE最大限度地降低了数据污染的风险，并提供了一种直接、高效的评估方式，这一点通过其与Chatbot Arena创意写作排名的强相关性（Spearman's ρ = 0.739, p < 0.001）在多种专有和开源模型上得到证实。对LLM与人类之间联想创造力的比较分析显示，虽然高性能LLM能达到与人类平均水平相当的分数，但专业人类始终优于LLM。此外，语言分析揭示，人类和LLM在联想中都表现出具体性下降的趋势，而人类展现出更多样化的联想模式。
英文摘要：
The evaluation of LLMs' creativity represents a crucial research domain, though challenges such as data contamination and costly human assessments often impede progress. Drawing inspiration from human creativity assessment, we propose PACE, asking LLMs to generate Parallel Association Chains to Evaluate their creativity. PACE minimizes the risk of data contamination and offers a straightforward, highly efficient evaluation, as evidenced by its strong correlation with Chatbot Arena Creative Writing rankings (Spearman's $ρ= 0.739$, $p < 0.001$) across various proprietary and open-source models. A comparative analysis of associative creativity between LLMs and humans reveals that while high-performing LLMs achieve scores comparable to average human performance, professional humans consistently outperform LLMs. Furthermore, linguistic analysis reveals that both humans and LLMs exhibit a trend of decreasing concreteness in their associations, and humans demonstrating a greater diversity of associative patterns.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
