arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2602.06801cs.LGcs.AI
收藏

大型语言模型中转向向量的不可识别性

On the Non-Identifiability of Steering Vectors in Large Language Models

Manipal Institute of Technology Bengaluru(马尼帕尔理工学院班加罗尔校区)

Sohan Venkatesh, Ashish Mahendran Kurapath

AI总结:

研究揭示大型语言模型中转向向量的不可识别性,指出在白盒单层访问下,转向向量因行为上无法区分的干预等价类而无法唯一确定,强调需超越行为测试的结构约束以实现可靠对齐干预。

摘要:

激活转向方法被广泛用于控制大型语言模型(LLM)的行为,并常被解释为揭示有意义的内部表示。这种解释假设转向方向是可识别且可从输入-输出行为中唯一恢复的。我们证明,在白盒单层访问下,转向向量本质上是不可识别的,因为存在大量行为上无法区分的干预等价类。实证上,我们发现正交扰动在多个模型和特性上实现了接近等效的效果,且影响大小可忽略。预训练语义分类器证实了在输出层面的等效性。我们通过激活协方差矩阵的SVD估计空域维度性,并验证等效性在整个操作相关的转向范围内具有鲁棒性。关键地,我们证明不可识别性是一种稳健的几何属性,能够在多样的提示分布中持续存在。这些发现揭示了基础可解释性限制,并突显了需要超越行为测试的结构约束以实现可靠对齐干预的必要性。

英文摘要:

Activation steering methods are widely used to control large language model (LLM) behavior and are often interpreted as revealing meaningful internal representations. This interpretation assumes that steering directions are identifiable and uniquely recoverable from input-output behavior. We show that, under white-box single-layer access, steering vectors are fundamentally non-identifiable due to large equivalence classes of behaviorally indistinguishable interventions. Empirically, we find that orthogonal perturbations achieve near-equivalent efficacy with negligible effect sizes across multiple models and traits, with pre-trained semantic classifiers confirming equivalence at the output level. We estimate null-space dimensionality via SVD of activation covariance matrices and validate that equivalence holds robustly throughout the operationally relevant steering range. Critically, we show that non-identifiability is a robust geometric property that persists across diverse prompt distributions. These findings reveal fundamental interpretability limits and highlight the need for structural constraints beyond behavioral testing to enable reliable alignment interventions.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑