GEOSTEER：大型语言模型中激活引导的测地线优化

中文摘要 AI 辅助

激活引导通过在推理时修改大型语言模型（LLM）的隐藏激活，提供了一种轻量级的模型控制方式。在这些方法中，保范引导旨在不改变激活范数的前提下改变模型行为，从而降低表示崩溃和性能退化的风险。然而，现有的保范方法受限于预定义的引导轨迹以及对单步更新的依赖，这可能无法捕捉激活分布的复杂结构。我们提出了GeoSteer，一种基于优化的保范激活引导方法。GeoSteer将引导问题形式化为黎曼优化问题，并通过在表示流形上执行一系列小步测地线更新来调整激活。为避免固定的引导方向，GeoSteer学习一个非线性的激活空间目标函数，以区分期望激活与非期望激活，并利用该函数自适应地指导每一步引导。这种多步公式带来了更平滑、更稳定且更一致的引导行为，同时保持激活范数不变。在TruthfulQA、RealToxicityPrompts和UltraFeedback基准测试中，GeoSteer持续优于最先进的激活引导基线。这些结果表明，通过用自适应的、几何感知的优化替代预定义的单步编辑，可以更有效地实现保范引导。

英文摘要

Activation steering provides a lightweight way to control large language models (LLMs) by modifying their hidden activations at inference time. Among these approaches, norm-preserving steering aims to change model behavior without altering the activation norm, reducing the risk of representation collapse and degradation. However, existing norm-preserving methods are limited by predefined steering trajectories and by their reliance on one-step updates, which may fail to capture the complex structure of activation distributions. We propose GeoSteer, an optimization-based method for norm-preserving activation steering. GeoSteer formulates steering as a Riemannian optimization problem and updates activations through a sequence of small geodesic steps on the representation manifold. To avoid fixed steering directions, GeoSteer learns a nonlinear activation-space objective that distinguishes desired from undesired activations, and uses this function to adaptively guide each steering step. This multistep formulation yields smoother, more stable, and more consistent steering behavior while preserving the activation norm. Across TruthfulQA, RealToxicityPrompts, and UltraFeedback benchmarks, GeoSteer consistently improves over state-of-the-art activation steering baselines. These results suggest that norm-preserving steering can be made more effective by replacing predefined one-step edits with adaptive, geometry-aware optimization.

发表机构

• University of Arkansas（阿肯色大学）

机构由 AI 辅助整理，请以论文原文为准。