天降还是地狱？评估大语言模型幻觉的智能与缺陷
Heaven-Sent or Hell-Bent? Benchmarking the Intelligence and Defectiveness of LLM Hallucinations
AI总结：
本文提出HIC-Bench框架，通过分类智能与缺陷幻觉，评估LLM在创造力与准确性之间的平衡，揭示幻觉对科学创新的推动作用。
摘要：
大语言模型（LLM）中的幻觉通常被视为需要最小化的错误。然而，近期的观点表明，某些幻觉可能编码了创造性的或知识上有价值的内容，这一维度在现有文献中仍未被量化。现有的幻觉检测方法主要集中在事实一致性上，难以处理异质的科学任务，并且难以在创造力与准确性之间取得平衡。为了解决这些挑战，我们提出了HIC-Bench，一种新的评估框架，将幻觉分为智能幻觉（IH）和缺陷幻觉（DH），使系统研究其在LLM创造力中的相互作用成为可能。HIC-Bench具有三个核心特征：（1）结构化的IH/DH评估。使用一个多维指标矩阵整合Torrance创造性思维测试（TTCT）指标（原创性、可行性、价值）与幻觉特定维度（科学合理性、事实偏差）；（2）跨领域适用性。涵盖十个科学领域，具有开放性创新任务；（3）动态提示优化。利用动态幻觉提示（DHP）引导模型朝向创造性和可靠性的输出。评估过程采用多个LLM法官，平均评分以减少偏见，人类注释者验证IH/DH分类。实验结果揭示了IH和DH之间非线性关系，证明了创造力和正确性可以共同优化。这些见解将IH定位为创造力的催化剂，并揭示了LLM幻觉驱动科学创新的能力。此外，HIC-Bench还提供了一个有价值的平台，用于推进对LLM幻觉创造性智能的研究。
英文摘要：
Hallucinations in large language models (LLMs) are commonly regarded as errors to be minimized. However, recent perspectives suggest that some hallucinations may encode creative or epistemically valuable content, a dimension that remains underquantified in current literature. Existing hallucination detection methods primarily focus on factual consistency, struggling to handle heterogeneous scientific tasks and balance creativity with accuracy. To address these challenges, we propose HIC-Bench, a novel evaluation framework that categorizes hallucinations into Intelligent Hallucinations (IH) and Defective Hallucinations (DH), enabling systematic investigation of their interplay in LLM creativity. HIC-Bench features three core characteristics: (1) Structured IH/DH Assessment. using a multi-dimensional metric matrix integrating Torrance Tests of Creative Thinking (TTCT) metrics (Originality, Feasibility, Value) with hallucination-specific dimensions (scientific plausibility, factual deviation); (2) Cross-Domain Applicability. spanning ten scientific domains with open-ended innovation tasks; and (3) Dynamic Prompt Optimization. leveraging the Dynamic Hallucination Prompt (DHP) to guide models toward creative and reliable outputs. The evaluation process employs multiple LLM judges, averaging scores to mitigate bias, with human annotators verifying IH/DH classifications. Experimental results reveal a nonlinear relationship between IH and DH, demonstrating that creativity and correctness can be jointly optimized. These insights position IH as a catalyst for creativity and reveal the ability of LLM hallucinations to drive scientific innovation.Additionally, the HIC-Bench offers a valuable platform for advancing research into the creative intelligence of LLM hallucinations.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
