arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2604.04020cs.CLcs.LG
收藏

揭示幻觉:从因果图注意力视角探讨大语言模型中的事实可靠性

Unmasking Hallucinations: A Causal Graph-Attention Perspective on Factual Reliability in Large Language Models

Sailesh kiran kurra, Shiek Ruksana, Vishal Borusu

AI总结:

本文提出因果图注意力网络框架,通过构建token级图来减少大语言模型中的幻觉问题,提升事实可靠性。

摘要:

本文主要聚焦于人工智能语言模型(LLMs)产生的幻觉问题。尽管LLMs在语言理解和生成方面表现出色,但其主要缺点是产生事实错误、误导或缺乏输入数据支持的输出。这些幻觉在医疗诊断或法律推理等场景中造成严重后果。本文提出因果图注意力网络(GCAN)框架,通过在Transformer架构中解读内部注意力流,结合自注意力权重和基于梯度的影响评分构建token级图,以减少幻觉。我们的方法使用新的度量标准——因果贡献分数(CCS)量化每个token的事实依赖性。我们进一步引入了一个事实锚定的图重加权层,动态减少易产生幻觉节点的影响。在TruthfulQA和HotpotQA等标准基准上的实验表明,与基线检索增强生成(RAG)模型相比,幻觉率减少了27.8%,事实准确性提高了16.4%。本文为未来LLM架构的可解释性、鲁棒性和事实可靠性做出了贡献。

英文摘要:

This paper primarily focuses on the hallucinations caused due to AI language models(LLMs).LLMs have shown extraordinary Language understanding and generation capabilities .Still it has major a disadvantage hallucinations which give outputs which are factually incorrect ,misleading or unsupported by input data . These hallucinations cause serious problems in scenarios like medical diagnosis or legal reasoning.Through this work,we propose causal graph attention network (GCAN) framework that reduces hallucinations through interpretation of internal attention flow within a transformer architecture with the help of constructing token level graphs that combine self attention weights and gradient based influence scores.our method quantifies each tokens factual dependency using a new metric called the Causal Contribution Score (CCS). We further introduce a fact-anchored graph reweighting layer that dynamically reduces the influence of hallucination prone nodes during generation. Experiments on standard benchmarks such as TruthfulQA and HotpotQA show a 27.8 percent reduction in hallucination rate and 16.4 percent improvement in factual accuracy over baseline retrieval-augmented generation (RAG) models. This work contributes to the interpretability,robustness, and factual reliability of future LLM architectures.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑