Sparse autoencoders (SAEs) have recently emerged as a powerful tool for language model steering. Prior work has explored top-k SAE latents for steering, but we observe that many dimensions among the top-k latents capture non-semantic features such as punctuation rather than semantic attributes like instructions. To address this, we propose focusing on a single, most relevant SAE latent (top-1), eliminating redundant features. We further identify a limitation in constant SAE steering, which often produces degenerate outputs such as repetitive single words. To mitigate this, we introduce a token-wise decaying steering strategy, enabling more faithful comparisons with mean activation difference baselines. Empirically, we show that steering an SAE latent associated with reasoning reliably elicits step-by-step mathematical reasoning and enhances inference quality, functionally resembling the effect of appending a guiding token. Our results demonstrate that SAEs outperform mean activation difference methods on mathematical reasoning benchmarks and match their performance on IF-Eval.
Multilingual Steering by Design: Multilingual Sparse Autoencoders and Principled Layer Selection
Saarland University(萨尔兰大学);German Research Center for Artificial Intelligence (DFKI)(德国人工智能研究中心);TU Darmstadt(德累斯顿技术大学);Centre for European Research in Trusted AI (CERTAIN)(欧洲可信AI研究中心)
Universal Boosts, Specific Suppressors: Sparse Autoencoder Steering of Medical Vision-Language Models
University of Zurich and University Hospital of Zurich(苏黎世大学及苏黎世大学医院);Kobe University(Kobe大学);ETH AI Center(苏黎世联邦理工学院人工智能中心);ETH Zurich(苏黎世联邦理工学院);Stanford University(斯坦福大学);Zurich University of Applied Sciences(苏黎世应用科学大学)
Interpretable Steering of Large Language Models with Feature Guided Activation Additions
Raffles Science Institute(莱佛士科学学院);Raffles Institution(莱佛士书院);Nous Research(努斯研究机构);Centre for Frontier AI Research (CFAR)(前沿人工智能研究中心);Agency for Science Technology and Research (A*STAR)(新加坡科技研究局)
LF-Steering: Latent Feature Activation Steering for Enhancing Semantic Consistency in Large Language Models
College of Intelligence and Computing, Tianjin University(天津大学智能与计算学部);IT Innovation and Research Center, Huawei Technologies(华为技术有限公司 IT 创新研究中心);Informatics, University of Edinburgh(爱丁堡大学信息学院)
Does higher interpretability imply better utility? A Pairwise Analysis on Sparse Autoencoders
School of Computing and Data Science, The University of Hong Kong(计算与数据科学学院,香港大学);School of Data Science, The Chinese University of Hong Kong, Shenzhen(数据科学学院,香港中文大学(深圳))
CLaS-Bench: A Cross-Lingual Alignment and Steering Benchmark
Saarland University(萨尔兰大学);German Research Center for Artificial Intelligence (DFKI)(德国人工智能研究中心(DFKI));Centre for European Research in Trusted AI (CERTAIN)(可信AI欧洲研究中心(CERTAIN));University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);TU Darmstadt(德累斯顿技术大学);Mila - Quebec Artificial Intelligence Institute(魁北克人工智能研究所(Mila));McGill University(麦吉尔大学)
Steering LVLMs via Sparse Autoencoder for Hallucination Mitigation
School of Computer Science and Engineering, Southeast University(东南大学计算机科学与工程学院);Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications (Southeast University)(东南大学新一代人工智能技术及其交叉应用关键实验室);Foundation Model Research Center, Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所基础模型研究中心);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);Department of Computer Science and Technology, Tsinghua University(清华大学计算机科学与技术系);Wuhan University of Technology(武汉理工大学);National University of Singapore(新加坡国立大学)
Causal Language Control in Multilingual Transformers via Sparse Feature Steering
University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);University of Maryland, College Park(马里兰大学学院公园分校);Barnard College(巴纳德学院);Algoverse AI Research(Algoverse人工智能研究);Meta FAIR Lab(Meta FAIR实验室)
Exploring and steering the moral compass of Large Language Models
Ernst Strüngmann Institute for Neuroscience(恩斯特·施特伦格曼神经科学研究所);Max Planck Society(马克斯·普朗克学会);Laboratory for Clinical Neuroscience(临床神经科学实验室);Centre for Biomedical Technology(生物医学技术中心);Universidad Politécnica de Madrid(马德里理工大学)