Large Language Models (LLMs) encode factual knowledge within hidden parametric spaces that are difficult to inspect or control. While Sparse Autoencoders (SAEs) can decompose hidden activations into more fine-grained, interpretable features, they often struggle to reliably align these features with human-defined concepts, resulting in entangled and distributed feature representations. To address this, we introduce AlignSAE, a method that aligns SAE features with a predefined ontology through a "pre-train, then post-train" curriculum. After an initial unsupervised training phase, we apply supervised post-training to bind specific concepts to dedicated latent slots while preserving the remaining capacity for general reconstruction. This separation creates an interpretable interface where specific concepts can be inspected and controlled without interference from unrelated features. Empirical results on GPT-2 across synthetic biographical QA (1-hop, 6 concepts) and compositional relational reasoning (2-hop, 20 concepts) demonstrate that AlignSAE enables precise causal interventions, such as reliable concept swaps (94.8% success in 1-hop), by targeting single, semantically aligned slots. AlignSAE achieves disentanglement improvements of $9$--$\infty\times$ over standard SAEs (measured via Mutual Information Gap), and further supports multi-hop reasoning and a mechanistic probe of grokking-like generalization dynamics. Although our evaluation focuses on GPT-2, AlignSAE itself is model-agnostic.
Regularized linear autoencoders recover the principal components, eventually
University of Toronto（多伦多大学）；Vector Institute（向量研究所）
近年来，我们对使用神经网络学习输入输出关系的理解得到了迅速提高，但对底层表示的收敛性知之甚少，即使是在线性自编码器（LAE）的简单情况下。我们表明，当使用适当的正则化进行训练时，LAEs可以直接学习最佳表示-有序，轴对齐的主成分。我们分析了两种这样的正则化方案：非均匀$\ell_2$正则化和嵌套dropout的确定性变体[Rippel et al，ICML' 2014]。虽然这两个正则化方案收敛到最佳表示，我们表明，这种收敛是缓慢的，由于病态，随着潜在的维数增加。我们表明，学习最佳表示的低效率是不可避免的-我们提出了一个简单的修改梯度下降更新，大大加快收敛经验。
Understanding Refusal in Language Models with Sparse Autoencoders
Nanyang Technological University（南洋理工大学）；Singapore University of Technology and Design（新加坡科技设计大学）；Digital Trust Centre（数字信任中心）；Institute of High Performance Computing (IHPC)（高性能计算研究所）
Physically-Constrained Autoencoder-Assisted Bayesian Optimization for Refinement of High-Dimensional Defect-Sensitive Single Crystalline Structure
Bredesen Center for Interdisciplinary Research, University of Tennessee, Knoxville, USA（跨学科研究中心，田纳西大学）；Center for Advanced Material Science and Manufacturing, University of Tennessee, Knoxville, TN 37996, USA（先进材料科学与制造中心，田纳西大学）；Department of Physics and Astronomy, University of Tennessee, Knoxville, TN 37996, USA（物理与天文学系，田纳西大学）；Georgia Institute of Technology, Atlanta, GA 30332, USA（佐治亚理工学院）；Virginia Polytechnic Institute and State University, Blacksburg, VA 24061, USA（弗吉尼亚理工大学）；Department of Materials Science and Engineering, University of Tennessee, Knoxville, TN 37996, USA（材料科学与工程系，田纳西大学）；Department of Physics and Astronomy, Institute for Quantum Matter, Johns Hopkins University, Baltimore, MD 21218（物理与天文学系，量子物质研究所，约翰霍普金斯大学）；Department of Materials Science and Engineering, Johns Hopkins University, Baltimore, MD 21218（材料科学与工程系，约翰霍普金斯大学）；The Ralph O’Connor Sustainable Energy Institute, Johns Hopkins University, Baltimore, MD 21218（罗伯特·奥康纳可持续能源研究所，约翰霍普金斯大学）；Advanced Photon Source, Argonne National Lab, Lemont, IL 60439, USA（先进光子源，阿贡国家实验室）；University of Tennessee-Oak Ridge Innovation Institute, Knoxville, TN 37996, USA（田纳西大学奥本创新研究所）
SLIM: Sparse Latent Steering for Interpretable and Property-Directed LLM-Based Molecular Editing
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；Nanjing University of Aeronautics and Astronautics（南京航空航天大学）；The 63rd Research Institute, National University of Defense Technology, Nanjing（国防科技大学第六三研究所，南京）
Stable and Steerable Sparse Autoencoders with Weight Regularization
Department of Biology, University of Oxford, Oxford, UK（牛津大学生物学系）；Kavli Institute for Nanoscience Discovery, University of Oxford, Oxford, UK（牛津大学纳科学发现研究所）；Department of Chemistry, University of Oxford, Oxford, UK（牛津大学化学系）
TEVI: Text-Conditioned Editing of Visual Representations via Sparse Autoencoders for Improved Vision-Language Alignment
Max Planck Institute for Informatics, Saarland Informatics Campus, Saarbrücken, Germany（马克斯·普朗克研究所信息学院，萨尔兰信息学院，德国萨尔布吕肯）；Department of Language Science and Technology, Saarland University, Saarbrücken, Germany（语言科学与技术系，萨尔兰大学，德国萨尔布吕肯）
Interpretable Spectral Variational AutoEncoder (ISVAE) for time series clustering
Universidad Carlos III de Madrid（马德里卡洛斯三世大学）；Instituto de Investigación Sanitaria Gregorio Marañón（格雷戈里奥·马拉尼翁卫生研究所）；Evidence Based Behaviour (eB2)（循证行为（eB2））；Oxford-Man Institute of Quantitative Finance, University of Oxford（牛津大学牛津-曼氏定量金融研究所）
Beyond Gaussian Bottlenecks: Topologically Aligned Encoding of Vision-Transformer Feature Spaces
Department of Computer Engineering, Koç University, Istanbul, Turkey（科克大学计算机工程系，伊斯坦布尔，土耳其）；Department of Computer Engineering, Hacettepe University, Ankara, Turkey（哈恰塔佩大学计算机工程系，安卡拉，土耳其）；KUIS AI Research Center, Istanbul, Turkey（KUIS人工智能研究中心，伊斯坦布尔，土耳其）；Department of Electrical and Electronics Engineering, Koç University, Istanbul, Turkey（科克大学电气与电子工程系，伊斯坦布尔，土耳其）
Conceptualizing Embeddings: Sparse Disentanglement for Vision-Language Models
Faculty of Mathematics and Computer Science, Jagiellonian University（雅盖隆大学数学与计算机科学学院）；Doctoral School of Exact and Natural Sciences, Jagiellonian University（雅盖隆大学精确与自然科学博士学校）；Centre for Credible AI, Warsaw University of Technology（华沙技术大学可信人工智能中心）
