arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.04718cs.LGcs.AIcs.CL
收藏

通过语言模型中几乎正交的特征实现孤立干预

Superposition Without Interference? Towards Isolated Interventions via Almost Orthogonal Features in Language Models

Max Planck Institute for Intelligent Systems(马克斯·普朗克智能系统研究所);ETH Zurich(苏黎世联邦理工学院);Max Planck ETH Center for Learning Systems(马克斯·普朗克-苏黎世联邦理工学院学习系统中心)

Moritz Miller, Florent Draye, Bernhard Schölkopf

AI总结:

该研究基于机械可解释性前提,针对语言模型特征纠缠问题,受“独立因果机制”启发,提出将内部特征约束为几乎正交,通过形式化问题、界定干扰传播并关联正则化,实现对数学推理概念更孤立的干预且保留模型性能。

摘要:

机械可解释性的一个核心前提是,语言模型中的有意义概念由激活空间中的线性特征表示。为使这些特征支持可靠干预,操纵一个特征不应大幅改变其他特征的效果。然而在实践中,特征纠缠会导致干扰,使局部干预产生意外的下游效应。受“独立因果机制”原则启发,我们提议将内部特征约束为几乎正交。我们认为这能促进适用于因果干预的模块化表示。我们通过根据特征干扰来刻画理想化孤立干预与其对模型输出的实际效果之间的差距,对该问题进行形式化。我们根据特征字典的自相干性对特征干扰的传播进行上限界定,并将这种差异与字典本身的显式正交正则化相关联。从经验上看,我们表明这种正则化能够在保留模型性能的同时,对数学推理概念进行更孤立的干预。我们的代码可在[this https URL]获取。

英文摘要:

A central premise in mechanistic interpretability is that meaningful concepts in language models are represented by linear features in activation space. For such features to support reliable interventions, manipulating one feature should not substantially alter the effects of others. In practice, however, feature entanglement leads to interference such that localized interventions can have unintended downstream effects. Motivated by the _Independent Causal Mechanisms_ principle, we propose to constrain internal features to be almost orthogonal. We argue that this promotes modular representations amenable to causal intervention. We formalize this problem by characterizing the gap between an idealized isolated intervention and its realized effect on model outputs in terms of feature interference. We upper-bound the propagation of feature interference in terms of the self-coherence of the feature dictionary, and relate this discrepancy to an explicit orthogonality regularization on the dictionary itself. Empirically, we show that this regularization enables more isolated interventions on mathematical reasoning concepts while preserving model performance. Our code is available under https://github.com/mrtzmllr/sae-icm.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑