HalluMat: 通过多阶段验证检测LLM生成的材料科学内容中的幻觉
HalluMat: Detecting Hallucinations in LLM-Generated Materials Science Content Through Multi-Stage Verification
AI总结：
HalluMat通过多阶段验证框架检测LLM生成的材料科学内容中的幻觉，减少30%的幻觉率并引入PHCS量化响应不一致性。
摘要：
人工智能（AI），特别是大语言模型（LLMs），正在转变科学发现，使快速的知识生成和假设提出成为可能。然而，一个关键挑战是幻觉，即LLMs生成事实性错误或误导性信息，影响研究的完整性。为此，我们引入HalluMatData，一个用于评估幻觉检测方法、事实一致性及AI生成材料科学内容响应鲁棒性的基准数据集。此外，我们提出HalluMatDetector，一个多阶段幻觉检测框架，整合了内在验证、多源检索、矛盾图分析和基于指标的评估，以检测和缓解LLM幻觉。我们的发现表明，幻觉水平在材料科学子领域中差异显著，高熵查询表现出更大的事实不一致性。通过利用HalluMatDetector验证流程，我们比标准LLM输出减少了30%的幻觉率。此外，我们引入了改写幻觉一致性分数（PHCS），以量化LLM响应在语义等价查询中的不一致性，提供对模型可靠性的深入见解。
英文摘要：
Artificial Intelligence (AI), particularly Large Language Models (LLMs), is transforming scientific discovery, enabling rapid knowledge generation and hypothesis formulation. However, a critical challenge is hallucination, where LLMs generate factually incorrect or misleading information, compromising research integrity. To address this, we introduce HalluMatData, a benchmark dataset for evaluating hallucination detection methods, factual consistency, and response robustness in AI-generated materials science content. Alongside this, we propose HalluMatDetector, a multi-stage hallucination detection framework that integrates intrinsic verification, multi-source retrieval, contradiction graph analysis, and metric-based assessment to detect and mitigate LLM hallucinations. Our findings reveal that hallucination levels vary significantly across materials science subdomains, with high-entropy queries exhibiting greater factual inconsistencies. By utilizing HalluMatDetector verification pipeline, we reduce hallucination rates by 30% compared to standard LLM outputs. Furthermore, we introduce the Paraphrased Hallucination Consistency Score (PHCS) to quantify inconsistencies in LLM responses across semantically equivalent queries, offering deeper insights into model reliability.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
