Linear probes and sparse autoencoders consistently recover meaningful structure from transformer representations -- yet why should such simple methods succeed in deep, nonlinear systems? We show this is not merely an empirical regularity but a consequence of architectural necessity: transformers communicate information through linear interfaces (attention OV circuits, unembedding matrices), and any semantic feature decoded through such an interface must occupy a context-invariant linear subspace. We formalize this as the \emph{Invariant Subspace Necessity} theorem and derive the \emph{Self-Reference Property}: tokens directly provide the geometric direction for their associated features, enabling zero-shot identification of semantic structure without labeled data or learned probes. Empirical validation in eight classification tasks and four model families confirms the alignment between class tokens and semantically related instances. Our framework provides \textbf{a principled architectural explanation} for why linear interpretability methods work, unifying linear probes and sparse autoencoders.
FADE: Why Bad Descriptions Happen to Good Features
Department of Artificial Intelligence, Fraunhofer Heinrich Hertz Institute（人工智能系，弗劳恩霍夫海因里希·赫兹研究所）；Department of Electrical Engineering and Computer Science, Technische Universität Berlin（电气工程与计算机科学系，柏林技术大学）；BIFOLD - Berlin Institute for the Foundations of Learning and Data（柏林学习与数据基础研究所）；Centre of eXplainable Artificial Intelligence, Technological University Dublin（可解释人工智能中心，都柏林技术大学）
Posterior-Calibrated Causal Circuits in Variational Autoencoders: Why Image-Domain Interpretability Fails on Tabular Data
Indian Institute of Technology, Patna（印度理工学院巴特那分校）；Rajarshi School of Management Technology, Varanasi（拉贾尔希管理技术学院瓦拉纳西分校）；Jaypee Institute of Information Technology, Noida（杰皮信息技术学院诺伊达分校）
Why Does It Look There? Structured Explanations for Image Classification
Department of Computer Science, Tulane University, New Orleans LA 70118, USA（计算机科学系，路易斯安那大学，新奥尔良 LA 70118, 美国）；Department of Biochemistry and Molecular Biology, Tulane University School of Medicine, New Orleans LA 70112, USA（生物化学与分子生物学系，路易斯安那大学医学院，新奥尔良 LA 70112, 美国）
Sandia National Laboratories（桑迪亚国家实验室）；New Mexico Institute of Mining and Technology（新墨西哥矿业理工大学）；Institute for Complex Additive Systems Analysis（复杂增材系统分析研究所）
Optimization and Interpretability of Graph Attention Networks for Small Sparse Graph Structures in Automotive Applications
CARISSMA Institute of Automated Driving, Technische Hochschule Ingolstadt（英戈尔施塔特应用技术大学CARISSMA自动驾驶研究所）；AUDI AG（奥迪股份公司）；Technical University of Munich（慕尼黑工业大学）
A Unified Theory of Sparse Dictionary Learning in Mechanistic Interpretability: Piecewise Biconvexity and Spurious Minima
National University of Singapore（新加坡国立大学）；Indian Institute of Technology, Dhanbad（印度德里理工学院）；Chinese University of Hong Kong（香港中文大学）；Hong Kong University of Science and Technology（香港科学与技术大学）；Peking University（北京大学）
Model Interpretability through the Lens of Computational Complexity
Institute for Mathematical and Computational Engineering, PUC-Chile（智利天主教大学数学与计算工程学院）；Inria Lille（里尔信息与自动化研究所）；Universidad de Chile（智利大学）；Millennium Institute for Foundational Research on Data（数据基础研究千年研究所）
Mechanistic Interpretability of EEG Foundation Models via Sparse Autoencoders
BrainCapture；DTU Health Tech（技术大学丹麦健康技术）；DTU Compute（技术大学丹麦计算）；Department of Biomedical Data Science（生物医学数据科学系）；Department of Computer Science（计算机科学系）；Seer Medical（Seer医疗）；Filadelfia Epilepsy Hospital（菲拉德尔菲亚癫痫医院）；University Hospital of Copenhagen（哥本哈根大学医院）
