arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.19239stat.MLcs.LG
收藏

语言模型中的注意力缺陷:对程序性幻觉的因果解释

Attention Deficits in Language Models: Causal Explanations for Procedural Hallucinations

Hassana Labs(Hassana 实验室);University of Oxford(牛津大学)

Ahmed Karim, Fatima Sheaib, Zein Khamis, Maggie Chlon, Jad Awada, Leon Chlon

AI总结:

语言模型在执行复杂流程时出现程序性幻觉,研究发现其源于回答模式未激活或候选选择偏差,通过干预可显著提升远距离绑定准确性。

摘要:

大型语言模型能够遵循复杂的流程,却在看似简单的最终步骤上失败:报告它们刚刚计算出的值。我们研究这种现象为程序性幻觉:即使正确值存在于上下文中,模型也未能执行可验证的、基于提示的规范。在具有已知单个token候选集的长上下文绑定任务中,我们发现许多错误是读出阶段路由失败。具体而言,错误分解为阶段2A(门控)错误,即模型未进入回答模式,以及阶段2B(绑定)错误,即进入回答模式但选择了错误的候选(通常由于最近性偏差)。在困难的领域,阶段2B在我们的任务中占大多数错误,跨模型家族(表1)。在阶段2B错误试验中,对最终层残差流的线性探针在正确值的恢复上远高于随机(例如,Qwen2.5-3B为74% vs. 2%;表2),表明答案被编码但未使用。我们通过可用性与使用性互信息以及伪先验干预正式化了“存在但未使用”,从而得到输出可计算的诊断和信息预算证书。最后,一个 oracle 检查点干预,即在查询附近重新陈述真实的绑定,可以几乎消除远距离的阶段2B失败(例如,Qwen2.5-3B在k=1024时从0/400变为399/400;表8)。

英文摘要:

Large language models can follow complex procedures yet fail at a seemingly trivial final step: reporting a value they themselves computed moments earlier. We study this phenomenon as \emph{procedural hallucination}: failure to execute a verifiable, prompt-grounded specification even when the correct value is present in context. In long-context binding tasks with a known single-token candidate set, we find that many errors are readout-stage routing failures. Specifically, failures decompose into Stage~2A (gating) errors, where the model does not enter answer mode, and Stage~2B (binding) errors, where it enters answer mode but selects the wrong candidate (often due to recency bias). In the hard regime, Stage~2B accounts for most errors across model families in our tasks (Table~1). On Stage~2B error trials, a linear probe on the final-layer residual stream recovers the correct value far above chance (e.g., 74\% vs.\ 2\% on Qwen2.5-3B; Table~2), indicating that the answer is encoded but not used. We formalize ``present but not used'' via available vs.\ used mutual information and pseudo-prior interventions, yielding output-computable diagnostics and information-budget certificates. Finally, an oracle checkpointing intervention that restates the true binding near the query can nearly eliminate Stage~2B failures at long distance (e.g., Qwen2.5-3B $0/400 \rightarrow 399/400$ at $k = 1024$; Table~8).

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑