当注意力关闭:LLMs在多轮交互中如何失去线索
When Attention Closes: How LLMs Lose the Thread in Multi-Turn Interaction
AI总结:
研究揭示LLMs在多轮对话中因注意力关闭导致任务目标丢失的现象,提出目标可访问性比率(GAR)及通道转换框架,分析不同架构下目标相关行为的生存机制及失败模式。
摘要:
大型语言模型在单轮中能遵循复杂指令,但在长多轮交互中常失去指令、角色和规则的线索。这种退化已通过行为测量,但未有机制解释。本文提出通道转换理论:目标定义词通过注意力变得不可达,而目标相关信息可能保留在残差表示中。引入目标可访问性比率(GAR),测量生成词对任务定义目标词的注意力,结合滑动窗口消融和残差流探针。当注意力指向指令关闭时,残余信息揭示了架构特性。不同架构下,转换导致不同失败模式:某些模型在注意力消失时仍保留目标条件行为,另一些模型即使有可解码的残余目标信息也失败,编码层从2到27不等。在Mistral模型中强制关闭注意力通道,使20事实记忆任务的回忆率从接近完美降至11%,且在无用户压力下,角色约束违规率超过对抗压力基线。线性探针从残余表示中恢复每回合回忆结果,AUC高达0.99。不同架构和模型规模下,注意力损失与残余解码性之间的差距预测目标相关行为在通道关闭后的生存情况。本文贡献了GAR作为诊断工具、通道转换框架作为受控机制解释,以及在滑动注意力关闭下的失败时间参数预测。
英文摘要:
Large language models can follow complex instructions in a single turn, yet over long multi-turn interactions they often lose the thread of instructions, persona, and rules. This degradation has been measured behaviorally but not mechanistically explained. We propose a channel-transition account: goal-defining tokens become less accessible through attention, while goal-related information may persist in residual representations. We introduce the Goal Accessibility Ratio (GAR), measuring attention from generated tokens to task-defining goal tokens, and combine it with sliding-window ablations and residual-stream probes. When attention to instructions closes, what survives reveals architecture. Across architectures, the transition yields qualitatively distinct failure modes: some models preserve goal-conditioned behavior at vanishing attention, others fail despite decodable residual goal information, and the layer at which this encoding emerges varies from 2 to 27. A within-model causal ablation that force-closes the attention channel in Mistral collapses recall from near-perfect to 11% on a 20-fact retention task and raises persona-constraint violations above an adversarial-pressure baseline without user pressure, with both effects emerging at the predictable crossover turn. Linear probes recover per-episode recall outcomes from residual representations with AUC up to 0.99 across all four primary architectures, while input embeddings remain at chance. Across architectures and model scales, the gap between attention loss and residual decodability predicts whether goal-conditioned behavior survives channel closure. We contribute GAR as a diagnostic, the channel-transition framework as a controlled mechanistic account, and a parametric prediction of failure timing under windowed attention closure.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。