莎士比亚式的火花：LLMs解码层中幻觉与创造力的共舞
Shakespearean Sparks: The Dance of Hallucination and Creativity in LLMs' Decoding Layers
AI总结：
本文提出评估框架HCL，定量分析LLMs解码层中幻觉与创造力的权衡关系，发现不同模型规模存在最佳平衡层且该层置信度更高。
摘要：
大型语言模型（LLMs）通常会产生幻觉，这一现象往往与创造力相关。尽管以往研究主要从理论或定性的角度探讨这种联系，但我们的工作采用定量方法，系统地考察了LLMs中幻觉与创造力之间的关系。鉴于创造力的复杂性，我们提出了一种针对LLMs的狭义定义，并引入了一个评估框架HCL，用于在解码过程中量化LLMs不同层的幻觉和创造力。我们的实证分析揭示，幻觉与创造力之间存在一种权衡关系，这种关系在层深度、模型类型和模型规模上保持一致。值得注意的是，在不同的模型架构中，我们在每个模型规模上都识别出了一个能够最佳平衡这种权衡的特定层。此外，最佳层往往出现在较大模型的较浅层中，并且模型在该层的置信度也显著更高。这些发现提供了一个定量视角，为LLMs创造力与幻觉之间的相互作用提供了新见解。我们的实验代码和数据可在 https://github.com/ZicongHe2002/HCL-Spark 获取。
英文摘要：
Large language models (LLMs) are known to hallucinate, a phenomenon often linked to creativity. While previous research has primarily explored this connection through theoretical or qualitative lenses, our work takes a quantitative approach to systematically examine the relationship between hallucination and creativity in LLMs. Given the complex nature of creativity, we propose a narrow definition tailored to LLMs and introduce an evaluation framework, HCL, which quantifies Hallucination and Creativity across different Layers of LLMs during decoding. Our empirical analysis reveals a tradeoff between hallucination and creativity that is consistent across layer depth, model type, and model size. Notably, across different model architectures, we identify a specific layer at each model size that optimally balances this tradeoff. Additionally, the optimal layer tends to appear in the early layers of larger models, and the confidence of the model is also significantly higher at this layer. These findings provide a quantitative perspective that offers new insights into the interplay between LLM creativity and hallucination. The code and data for our experiments are available at https://github.com/ZicongHe2002/HCL-Spark.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
