Sparse dictionary learning has been a rapidly growing technique in mechanistic interpretability to attack superposition and extract more human-understandable features from model activations. We ask a further question based on the extracted more monosemantic features: How do we recognize circuits connecting the enormous amount of dictionary features? We propose a circuit discovery framework alternative to activation patching. Our framework suffers less from out-of-distribution and proves to be more efficient in terms of asymptotic complexity. The basic unit in our framework is dictionary features decomposed from all modules writing to the residual stream, including embedding, attention output and MLP output. Starting from any logit, dictionary feature or attention score, we manage to trace down to lower-level dictionary features of all tokens and compute their contribution to these more interpretable and local model behaviors. We dig in a small transformer trained on a synthetic task named Othello and find a number of human-understandable fine-grained circuits inside of it.
A Unified Theory of Sparse Dictionary Learning in Mechanistic Interpretability: Piecewise Biconvexity and Spurious Minima
National University of Singapore(新加坡国立大学);Indian Institute of Technology, Dhanbad(印度德里理工学院);Chinese University of Hong Kong(香港中文大学);Hong Kong University of Science and Technology(香港科学与技术大学);Peking University(北京大学)
Mechanistic Interpretability for Neural TSP Solvers
University of Washington, Michael G. Foster School of Business(华盛顿大学,迈克尔·G·福斯特商学院);University of Washington, Paul G. Allen School of Computer Science & Engineering(华盛顿大学,保罗·G·阿伦计算机科学与工程学院)
Mechanistic Interpretability of EEG Foundation Models via Sparse Autoencoders
BrainCapture;DTU Health Tech(技术大学丹麦健康技术);DTU Compute(技术大学丹麦计算);Department of Biomedical Data Science(生物医学数据科学系);Department of Computer Science(计算机科学系);Seer Medical(Seer医疗);Filadelfia Epilepsy Hospital(菲拉德尔菲亚癫痫医院);University Hospital of Copenhagen(哥本哈根大学医院)
CLT-Forge: A Scalable Library for Cross-Layer Transcoders and Attribution Graphs
Max Planck Institute for Intelligent Systems(马克斯·普朗克智能系统研究所);Jinesis AI Lab(Jinesis人工智能实验室);University of Toronto(多伦多大学);Vector Institute(向量研究所);CMU(卡内基梅隆大学);EuroSafeAI;ELLIS Institute Tübingen(图宾根ELLIS研究所)
ScopeSAE: Model-Scope Feature Discovery with Interpretable Layer Selection
The University of Sydney(悉尼大学);The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));DeepWisdom(深度智慧);University of Technology Sydney(悉尼科技大学)