通过注意力模式和语义对齐检测图检索增强生成中的幻觉
Detecting Hallucinations in Graph Retrieval-Augmented Generation via Attention Patterns and Semantic Alignment
AI总结：
本文提出通过注意力模式和语义对齐检测GraphRAG中的幻觉，开发了轻量级检测器GGA，提升了系统可靠性。
摘要：
基于图的检索增强生成（GraphRAG）通过将来自知识图的线性化子图检索到的外部知识整合到大型语言模型（LLMs）中，从而增强其能力。然而，LLMs在处理这些输入中的关系和拓扑信息时存在困难，导致与检索知识不一致的幻觉。为了分析LLMs在生成过程中如何关注和保留结构化知识，我们提出了两个轻量级的可解释性度量标准：路径依赖度（PRD），用于衡量对最短路径三元组的过度依赖，以及语义对齐分数（SAS），用于评估模型内部表示与检索知识的匹配程度。通过在基于知识的问答任务上的实证分析，我们识别出与过度依赖显著路径和弱语义基础相关的失败模式，这些模式通过高PRD和低SAS分数体现。我们进一步开发了一个轻量级的后处理幻觉检测器Graph Grounding and Alignment（GGA），在AUC和F1指标上优于强语义和置信度基线。通过将幻觉分析建立在机制性可解释性基础上，我们的工作提供了关于LLM结构限制如何导致幻觉的见解，为未来更可靠的GraphRAG系统设计提供了指导。
英文摘要：
Graph-based Retrieval-Augmented Generation (GraphRAG) enhances Large Language Models (LLMs) by incorporating external knowledge from linearized subgraphs retrieved from knowledge graphs. However, LLMs struggle to interpret the relational and topological information in these inputs, resulting in hallucinations that are inconsistent with the retrieved knowledge. To analyze how LLMs attend to and retain structured knowledge during generation, we propose two lightweight interpretability metrics: Path Reliance Degree (PRD), which measures over-reliance on shortest-path triples, and Semantic Alignment Score (SAS), which assesses how well the model's internal representations align with the retrieved knowledge. Through empirical analysis on a knowledge-based QA task, we identify failure patterns associated with over-reliance on salient paths and weak semantic grounding, as indicated by high PRD and low SAS scores. We further develop a lightweight post-hoc hallucination detector, Graph Grounding and Alignment (GGA), which outperforms strong semantic and confidence-based baselines across AUC and F1. By grounding hallucination analysis in mechanistic interpretability, our work offers insights into how structural limitations in LLMs contribute to hallucinations, informing the design of more reliable GraphRAG systems in the future.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
