发表机构
Islamic University of Technology; Systems and Software Lab (SSL)（伊斯兰科技大学; 系统与软件实验室）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
研究大语言模型激活空间控制问题时，提出基于统计的稀疏特征干预方法，先经可靠性过滤，再用三个统计量排序特征，构建引导方向。在多模型、领域和配置上实验，发现该方法有特定领域转移效果，且引导受多种因素影响，强调评估应兼顾质量与行为转移。
AI 中文摘要
激活引导为大语言模型行为控制提供了一种轻量级的微调替代方案，但基于SAE的引导方法通常依赖于学习到的引导目标或单标准特征选择。我们引入了一个透明的SAE特征引导管道，首先应用六条件可靠性过滤器，然后通过对F检验、KSG互信息和Cohen's d这三个互补统计量进行无加权博尔达共识来对稀疏特征进行排序。最终的引导方向被构建为SAE解码器行的Cohen's d加权组合，在近似SAE特征去相关的情况下提供了一个由Fisher-LDA激发的无优化方向。在三个Gemma系列模型、四个行为领域和356层强度配置上，该方法产生了可测量的特定领域转移，同时揭示了原始属性移动和质量保持生成之间的巨大差距。在最强配置下，逻辑正确性引导在Gemma 2 9B中达到了+1.16的主要分数增量；然而，我们更广泛的发现是，可用的引导在模型、领域、层和强度方面高度局部化。这些结果表明，激活引导评估应报告质量条件下的成功以及原始行为转移。我们的代码和数据可在该https URL上获取。
英文摘要
Activation steering adds a residual-stream direction at inference time, providing lightweight behavioral control without fine-tuning. Sparse autoencoders (SAEs) can make such interventions auditable by decomposing dense activations into an approximately monosemantic feature basis. We introduce SAE-StatSteer, a transparent, optimization-free pipeline. It first filters features through six reliability conditions, then ranks the survivors by an unweighted Borda consensus over three statistics, an $F$-test, KSG mutual information, and Cohen's $d$, and finally combines the selected SAE decoder rows using Cohen's-$d$ weights. We evaluate three Gemma-family models across four behavioral domains against seven dense or SAE-based baselines. Our quality-conditioned protocol requires attribute movement while preserving relevance, richness, and coherence. Raw success systematically overstates usable control because strong shifts often degrade generation quality, and effective steering is not governed by a universal layer or strength. SAE-StatSteer remains competitive with optimization-based methods while exposing every selection and weighting decision for audit. These results motivate reporting quality-conditioned success alongside raw behavioral shift. Our code and data are available at https://github.com/Oshayer-Siddique/LLM-Steering-Using-SAE.
CommentsUnder review, 26 pages, 5 figures, 16 tables
