概念接地注意力：图注入注意力、时间版本化与认知状态的控制评估

中文摘要 AI 辅助

知识密集型语言模型系统通常将外部知识表示为文本块或静态图，对概念演化、时间点推理以及已验证知识与推断知识之间的区分支持有限。我们引入了概念生命周期模型（CLM），该模型将概念表示为持久的、图接地的、时间版本化的实体，并带有明确的来源和认知状态；以及概念接地注意力（CGA），它通过图偏置自注意力（形式A）和概念节点上的门控交叉注意力（形式B）将概念图结构注入Transformer计算。我们在受控设置中使用禁用机制基线评估该框架。在200个MuSiQue和HotpotQA问题且检索固定的情况下，概念图检索恢复了显式的多跳路径，但并未提高证据召回率。形式A似乎引导了注意力，对黄金概念而非干扰概念的注意力是2.76倍，但禁用形式A时也出现了相同的比率；学习到的偏置可忽略不计，且没有答案发生变化。保身份的形式B将F1从0.188提高到0.221，但控制概念得到0.213，表明大部分增益反映了新增容量。在LongMemEval上，显式时间表示在所有测试的生成器（高达122B参数）上将答案准确率提高了13到25个百分点，而简化的CLM版本解析与日期序列化表现相似，因为概念身份未被可靠建立。在合成源独立性任务中，协议派生的认知状态将微调小模型中无依据断言从28%降至0.1%，在72-122B模型中从19-68%降至0-5%。总体而言，结果支持将时间有效性和认知状态显式化，同时表明没有禁用机制控制，图注意力诊断不具有信息性。

英文摘要

Knowledge-intensive language-model systems typically represent external knowledge as text chunks or static graphs, with limited support for concept evolution, point-in-time reasoning, and distinctions between validated and inferred knowledge. We introduce the Concept Lifecycle Model (CLM), which represents concepts as persistent, graph-grounded, temporally versioned entities with explicit provenance and epistemic status, and Concept-Grounded Attention (CGA), which injects concept-graph structure into transformer computation through graph-biased self-attention (Form A) and gated cross-attention over concept nodes (Form B). We evaluate the framework in controlled settings using disabled-mechanism baselines. On 200 MuSiQue and HotpotQA questions with retrieval fixed, concept-graph retrieval recovers explicit multi-hop paths but does not improve evidence recall. Form A appears to steer attention, with 2.76 times more attention on gold than distractor concepts, but the same ratio occurs when Form A is disabled; the learned bias is negligible and no answers change. An identity-preserving Form B improves F1 from 0.188 to 0.221, but control concepts yield 0.213, indicating that most of the gain reflects added capacity. On LongMemEval, explicit temporal representation improves answer accuracy by 13 to 25 points across all tested generators, up to 122B parameters, while simplified CLM version resolution performs similarly to dated serialization because concept identity is not established reliably. On a synthetic source-independence task, protocol-derived epistemic status reduces unsupported assertions from 28% to 0.1% in a fine-tuned small model and from 19-68% to 0-5% in 72-122B models. Overall, the results support making temporal validity and epistemic status explicit, while showing that graph-attention diagnostics are not informative without disabled-mechanism controls.

发表机构

• SeKondBrain AI Labs（SeKondBrain AI 实验室）

机构由 AI 辅助整理，请以论文原文为准。