迈向可靠的科学假设生成：评估大型语言模型的真实性与幻觉
Toward Reliable Scientific Hypothesis Generation: Evaluating Truthfulness and Hallucination in Large Language Models
AI总结：
本文提出TruthHypo基准与基于知识的幻觉检测器KnowHD，用于评估并筛选LLM生成的真实科学假设，实验与人工评估验证了其有效性。
摘要：
大型语言模型（LLM）已在生物医学等科学学科中展现出巨大潜力，尤其是在假设生成方面：它们能够分析大量文献、识别模式并提出研究方向。然而，一个关键挑战在于评估所生成假设的真实性，因为验证其准确性通常需要大量时间与资源。此外，LLM中的幻觉问题可能导致生成看似合理但最终错误的假设，从而削弱其可靠性。为促进对这些挑战的系统研究，我们提出了TruthHypo，这是一个用于评估LLM生成真实科学假设能力的基准；同时提出KnowHD，这是一种基于知识的幻觉检测器，用于评估假设在多大程度上扎根于现有知识。我们的结果表明，LLM难以生成真实的假设。通过分析推理步骤中的幻觉，我们证明KnowHD提供的扎根性分数可作为有效指标，用于从LLM的多样输出中筛选真实假设。人工评估进一步验证了KnowHD在识别真实假设和加速科学发现方面的效用。我们的数据和源代码可在https://github.com/Teddy-XiongGZ/TruthHypo获取。
英文摘要：
Large language models (LLMs) have shown significant potential in scientific disciplines such as biomedicine, particularly in hypothesis generation, where they can analyze vast literature, identify patterns, and suggest research directions. However, a key challenge lies in evaluating the truthfulness of generated hypotheses, as verifying their accuracy often requires substantial time and resources. Additionally, the hallucination problem in LLMs can lead to the generation of hypotheses that appear plausible but are ultimately incorrect, undermining their reliability. To facilitate the systematic study of these challenges, we introduce TruthHypo, a benchmark for assessing the capabilities of LLMs in generating truthful scientific hypotheses, and KnowHD, a knowledge-based hallucination detector to evaluate how well hypotheses are grounded in existing knowledge. Our results show that LLMs struggle to generate truthful hypotheses. By analyzing hallucinations in reasoning steps, we demonstrate that the groundedness scores provided by KnowHD serve as an effective metric for filtering truthful hypotheses from the diverse outputs of LLMs. Human evaluations further validate the utility of KnowHD in identifying truthful hypotheses and accelerating scientific discovery. Our data and source code are available at https://github.com/Teddy-XiongGZ/TruthHypo.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
