arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2605.14749cs.CLcs.AIcs.LG
收藏

非线性干预大语言模型

Non-linear Interventions on Large Language Models

Department of Linguistics, Seoul National University, Republic of Korea(韩国首尔国立大学语言系)

Sangwoo Kim

AI总结:

本文提出非线性干预方法,用于处理大语言模型中非线性特征,通过隐式特征干预提升拒绝绕过性能。

摘要:

干预是理解大语言模型内部表示最代表性且广泛应用的方法。然而,现有干预方法受限于线性表示假设,无法触及非线性流形上的特征。本文提出一种通用的干预框架,自然扩展到非线性表示特征,并引入学习过程以实现隐式特征的干预。我们在拒绝绕过引导任务中验证了框架,通过干预控制拒绝的非线性特征,比线性基线更精确。

英文摘要:

Intervention is one of the most representative and widely used methods for understanding the internal representations of large language models (LLMs). However, existing intervention methods are confined to linear interventions grounded in the Linear Representation Hypothesis, leaving features encoded along non-linear manifolds beyond their reach. In this work, we introduce a general formulation of intervention that extends naturally to non-linearly represented features, together with a learning procedure that further enables intervention on implicit features lacking a direct output signature. We validate our framework on refusal bypass steering, where it steers the model more precisely than linear baselines by intervening on a non-linear feature governing refusal.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑