Fact Grounded Attention：通过注意力层级知识整合消除大语言模型中的幻觉
Fact Grounded Attention: Eliminating Hallucination in Large Language Models Through Attention Level Knowledge Integration
AI总结：
针对大语言模型易产生事实幻觉的问题，提出事实锚定注意力（FGA）架构，通过在Transformer注意力分数层注入可验证知识，在多领域技术查询实验中大幅提升准确率，且知识更新无需重训、耗时极短，实现可验证事实的幻觉彻底消除。
摘要：
“知识最大的敌人不是无知，而是知识的错觉。”大语言模型已攻克自然语言处理难题，但仍受限于自身的概率生成特性——会自信地编造其从未真正掌握的事实。我们提出Fact Grounded Attention（FGA，事实锚定注意力），这是一种新型架构改进方案，通过将可验证知识直接注入注意力机制，将不可靠的语言模型转化为确定性的事实输出模型。现有方法要么在生成后修补幻觉问题，要么在输入前拼接检索到的文本，而FGA不同，它介入Transformer的数学核心——softmax前的注意力分数，构建出当知识库中存在对应事实时绝不会产生幻觉的模型。我们在涵盖智能手机、笔记本电脑和电动汽车领域的1107条技术查询上开展实验，结果表明，原生Llama 3.2的准确率仅为6.3%，而搭载FGA后准确率提升至99.7%。更关键的是，知识更新无需重新训练，耗时不到1秒，而参数编辑类方法需要数小时。FGA不只是减少幻觉——对于可验证事实，它能彻底消除幻觉，标志着神经语言生成从概率近似到确定性精准输出的根本性转变。
英文摘要：
"The greatest enemy of knowledge is not ignorance, it is the illusion of knowledge." Large Language Models have conquered natural language but remain prisoners of their own probabilistic nature--confidently hallucinating facts they never truly knew. We present Fact Grounded Attention (FGA), a novel architectural modification that transforms unreliable language models into deterministic truth tellers by injecting verifiable knowledge directly into the attention mechanism. Unlike existing approaches that patch hallucinations after generation or prepend retrieved text, FGA intervenes at the mathematical heart of the transformer--the pre-softmax attention scores--creating a model that cannot hallucinate when facts exist in its knowledge base. Our experiments across 1,107 technical queries spanning smartphones, laptops, and electric vehicles demonstrate a transformation from 6.3% accuracy in vanilla Llama 3.2 to 99.7% accuracy with FGA. More critically, knowledge updates occur in under one second without retraining, compared to hours for parameter editing approaches. FGA doesn't just reduce hallucination--it eliminates it entirely for verifiable facts, marking a fundamental shift from probabilistic approximation to deterministic precision in neural language generation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
