沉默中的错误思考:对连续潜在推理的后门攻击
Thinking Wrong in Silence: Backdoor Attacks on Continuous Latent Reasoning
AI总结:
本文提出ThoughtSteer攻击,通过扰动输入嵌入向量,在连续潜在空间中产生可靠答案,同时规避所有现有防御,揭示了神经坍缩机制在潜在空间中的作用。
摘要:
新一代语言模型完全在连续隐藏状态中推理,不生成令牌且无审计轨迹。我们证明这种沉默创造了全新的攻击面。ThoughtSteer扰动输入层的单个嵌入向量,模型自身的多轮推理将扰动放大为被劫持的潜在轨迹,可靠地生成攻击者选择的答案,同时在所有token级防御中保持结构不可见。在两种架构(Coconut和SimCoT)、三个推理基准和从124M到3B参数的模型规模上,ThoughtSteer在接近基线的干净准确性下达到99%以上的攻击成功率,能够转移至未见基准无需重新训练(94-100%),规避所有五种评估的主动防御,并在25轮干净微调后仍能存活。这些结果归因于一个统一的机制:潜在空间中的神经坍缩将触发的表示拉向一个紧密的几何吸引子,解释了为何防御失败以及为何任何有效的后门必须留下线性可分的签名(探测AUC≥0.999)。然而,一个显著的悖论出现:即使模型输出错误答案,单个潜在向量仍编码正确答案。对抗信息不在于任何单个向量,而在于集体轨迹,使后门扰动成为连续推理机制性可解释性的新视角。代码和检查点可供获取。
英文摘要:
A new generation of language models reasons entirely in continuous hidden states, producing no tokens and leaving no audit trail. We show that this silence creates a fundamentally new attack surface. ThoughtSteer perturbs a single embedding vector at the input layer; the model's own multi-pass reasoning amplifies this perturbation into a hijacked latent trajectory that reliably produces the attacker's chosen answer, while remaining structurally invisible to every token-level defense. Across two architectures (Coconut and SimCoT), three reasoning benchmarks, and model scales from 124M to 3B parameters, ThoughtSteer achieves >=99% attack success rate with near-baseline clean accuracy, transfers to held-out benchmarks without retraining (94-100%), evades all five evaluated active defenses, and survives 25 epochs of clean fine-tuning. We trace these results to a unifying mechanism: Neural Collapse in the latent space pulls triggered representations onto a tight geometric attractor, explaining both why defenses fail and why any effective backdoor must leave a linearly separable signature (probe AUC>=0.999). Yet a striking paradox emerges: individual latent vectors still encode the correct answer even as the model outputs the wrong one. The adversarial information is not in any single vector but in the collective trajectory, establishing backdoor perturbations as a new lens for mechanistic interpretability of continuous reasoning. Code and checkpoints are available.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。