基于稀疏自编码器（SAE）的 steering 增强：邻居集成特征选择
Enhancing SAE-based Steering via Neighbor Integrated Feature Selection
查看机构详情
- The University of Hong Kong（香港大学）
- University of Science and Technology of China（中国科学技术大学）
机构由 AI 辅助整理，请以论文原文为准。
浏览论文内容
中文总结 AI 辅助
本文针对现有基于 SAE 的 steering 方法采用 top-k 特征选择易遗漏有效特征的问题，提出 NIFS 策略，通过利用表征相似性改进特征选择，在多项任务上实现性能提升。
中文摘要 AI 辅助
稀疏自编码器（SAE）将模型激活解耦为可解释特征，广泛用于大语言模型的 steering 调控。现有多数基于 SAE 的 steering 方法采用基于统计分数的 top-k 筛选特征，假设分数越高的特征 steering 效果越强，但本文证明该假设常不成立，导致特征选择次优。分析发现，有效 steering 特征可能分布在 SAE 特征拆分产生的表征相邻、语义相似的组中，组内特征虽 steering 影响力相当，但统计分数差异大，基于分数的选择会遗漏重要特征。基于此，本文提出可即插即用的邻居集成特征选择（Neighbor Integrated Feature Selection，NIFS）策略，利用表征相似性改进 steering 的特征选择。在多种基于 SAE 的 steering 方法和任务上评估 NIFS，结果显示其相比传统 top-k 选择可实现一致的性能提升。
英文摘要
Sparse autoencoders (SAEs) disentangle model activations into interpretable features and are widely used for steering large language models. Most existing SAE-based steering methods select features by applying a top- filter based on statistical scores, assuming that higher-scoring features yield stronger steering effects. In this paper, we show that this assumption is often invalid, leading to suboptimal feature selection. Our analysis reveals that effective steering features may be distributed among representationally adjacent, semantically similar groups induced by feature splitting in SAEs. Within such groups, features may exhibit disparate statistical scores despite having comparable steering influence, causing score-based selection to overlook important features. Based on these observations, we propose \textsc{Neighbor Integrated Feature Selection} (\textsc{NIFS}), a plug-and-play strategy that leverages representation similarity to improve feature selection for steering. We evaluate \textsc{NIFS} across multiple SAE-based steering methods and tasks, and demonstrate consistent performance gains over conventional top-$k$ selection.
