arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2610.21708cs.LGcs.AIcs.CL
收藏

人类干预作为标签:定位、引导及(有时)编辑大语言模型智能体故障背后的计算过程

Human Interventions as Labels: Localizing, Steering, and (Sometimes) Editing the Computations Behind LLM Agent Failures

Università di Pisa(比萨大学)

Cesare Zavattari, Alessandro Tommasi, Giuseppe Prencipe

AI总结:

该研究将人类对LLM智能体故障的干预视为标签,在5个符合条件的模型中定位到提问与行动决策的因果位置,揭示过度提问的两种机制,提出引导方法并验证修复效果。

摘要:

当大语言模型(LLM)智能体出现异常行为时,人类会介入,而这种干预通常会被丢弃或作为训练数据重复使用。相反,我们将其视为一种测量:一个指向故障内部计算过程的标签。我们在受控的智能体环境中研究这一问题,针对一种故障(当请求已明确时仍要求澄清),使用基于规则的监督器生成标签,该标签已通过盲法人类判断验证。研究对象涵盖符合条件的模型,这些模型既会过度提问,有时也会对匹配的明确请求采取行动:13个模型中有5个符合条件(Qwen2.5-3B、gemma-2-2b、OLMo-3-7B、Mistral-Nemo-12B、Phi-4)。研究得出四项发现:第一,提问与行动的决策在每个符合条件的模型家族中都具有因果可定位性:从干预标签中学习到的方向,在确定的残差流深度下可翻转72%-100%的保留样本过度提问,该位置和剂量是基于训练数据选择的,而采用相同程序的随机方向最多只能翻转1个样本。第二,过度提问有两种机制特征:在四个模型家族中,模型明显知道请求是明确的(该信念可解码、因果惰性且与决策正交),但仍会提问;在第五个家族中,罕见的过度提问追踪不同的世界表征。第三,基于探测读数的引导可消除大部分模拟监督负担,同时保留合法提问(自举95%置信区间排除零);而提示指令会使情况恶化。第四,将修复措施整合到权重中会产生一种映射而非规则:在一个模型中,局限于因果带的LoRA通过了所有预先设定的标准,而在相同数据上进行行为克隆会破坏保留效果;在三个家族中则相反;在第五个家族中,结果会因随机种子而跨越边界。预先制定的用于预测该机制的几何规则在两次跨家族测试中失败;我们报告该失败,而非拟合替代规则。

英文摘要:

When an LLM agent misbehaves, a human steps in, and that intervention is usually discarded or replayed as training data. We treat it instead as a measurement: a label pointing at the internal computation that failed. We study it in a controlled agent environment, on one failure (asking a clarifying question when the request was already unambiguous), with labels from a rule-based overseer validated against blind human judgment. Claims cover eligible models, which both over-ask and sometimes act on matched unambiguous requests: five of thirteen qualified (Qwen2.5-3B, gemma-2-2b, OLMo-3-7B, Mistral-Nemo-12B, Phi-4). Four findings. First, the ask-vs-act decision is causally localizable in every eligible family: a direction learned from intervention labels flips 72-100% of held-out over-asks at a definite residual-stream depth, with site and dose chosen on training data, and random directions under the same procedure flip at most one episode. Second, over-asking has two mechanistic signatures: in four families the model demonstrably knows the request is unambiguous (the belief is decodable, causally inert and orthogonal to the decision) and asks anyway; in the fifth, the rare over-asks track different world-representations. Third, steering gated on a probe readout removes most of the simulated oversight burden while legitimate questions survive (bootstrap 95% intervals exclude zero); a prompt instruction makes it worse. Fourth, consolidating the fix into the weights yields a map, not a rule: on one model a LoRA confined to the causal band passes every criterion fixed in advance while behavior cloning on identical data breaks retention; on three families the reverse holds; on the fifth the outcome crosses the boundary across seeds. A geometric rule written down in advance to predict the regime failed on two out-of-family tests; we report the failure and stop rather than fit a replacement.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑