曲球操控：引导的正确方向并不总是线性
Curveball Steering: The Right Direction To Steer Isn't Always Linear
AI总结：
本文提出曲球操控方法，通过多项式核PCA在特征空间中进行干预，更尊重学习到的激活几何，优于线性PCA方法，尤其在几何扭曲强的场景中表现更佳。
摘要：
激活操控是通过干预内部表示来控制大语言模型（LLM）行为的常用方法。现有方法大多依赖线性表示假设，假设行为属性可通过全局线性方向操控。然而，实践中这种线性干预往往表现不一致。本文通过分析LLM激活空间的内在几何，质疑这一假设。通过测量几何扭曲（通过测地距离与欧几里得距离的比率），发现存在显著且概念依赖的扭曲，表明激活空间不能很好地用全局线性几何近似。受此启发，我们提出“曲球操控”，一种基于多项式核PCA的非线性操控方法，在特征空间中进行干预，更尊重学习到的激活几何。曲球操控在几何扭曲强的场景中优于线性PCA基于的操控，表明几何感知的非线性操控为全局线性干预提供了原理性替代方案。
英文摘要：
Activation steering is a widely used approach for controlling large language model (LLM) behavior by intervening on internal representations. Existing methods largely rely on the Linear Representation Hypothesis, assuming behavioral attributes can be manipulated using global linear directions. In practice, however, such linear interventions often behave inconsistently. We question this assumption by analyzing the intrinsic geometry of LLM activation spaces. Measuring geometric distortion via the ratio of geodesic to Euclidean distances, we observe substantial and concept-dependent distortions, indicating that activation spaces are not well-approximated by a globally linear geometry. Motivated by this, we propose "Curveball steering", a nonlinear steering method based on polynomial kernel PCA that performs interventions in a feature space, better respecting the learned activation geometry. Curveball steering consistently outperforms linear PCA-based steering, particularly in regimes exhibiting strong geometric distortion, suggesting that geometry-aware, nonlinear steering provides a principled alternative to global, linear interventions.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
