Large Language Models (LLMs) have experienced a rapid rise in AI, changing a wide range of applications with their advanced capabilities. As these models become increasingly integral to decision-making, the need for thorough interpretability has never been more critical. Mechanistic Interpretability offers a pathway to this understanding by identifying and analyzing specific sub-networks or 'circuits' within these complex systems. A crucial aspect of this approach is Automated Circuit Discovery, which facilitates the study of large models like GPT4 or LLAMA in a feasible manner. In this context, our research evaluates a recent method, Brain-Inspired Modular Training (BIMT), designed to enhance the interpretability of neural networks. We demonstrate how BIMT significantly improves the efficiency and quality of Automated Circuit Discovery, overcoming the limitations of manual methods. Our comparative analysis further reveals that BIMT outperforms existing models in terms of circuit quality, discovery time, and sparsity. Additionally, we provide a comprehensive computational analysis of BIMT, including aspects such as training duration, memory allocation requirements, and inference speed. This study advances the larger objective of creating trustworthy and transparent AI systems in addition to demonstrating how well BIMT works to make neural networks easier to understand.
When the Coffee Feature Activates on Coffins: An Analysis of Feature Extraction and Steering for Mechanistic Interpretability
Department of Philosophy of Nature and Technology(自然哲学与技术系);Munich School of Philosophy(慕尼黑哲学学院);Division of the Humanities and Social Sciences(人文与社会科学系);California Institute of Technology(加州理工学院)
MINAR: Mechanistic Interpretability for Neural Algorithmic Reasoning
Pacific Northwest National Laboratory, Richland, WA(太平洋西北国家实验室);Halıcıoğlu Data Science Institute, University of California, San Diego, San Diego, CA(哈利奇奥格鲁数据科学研究所,加州大学圣地亚哥分校);Department of Computer and Information Science, University of Pennsylvania, Pennsylvaina, PA(计算机与信息科学系,宾夕法尼亚大学);Department of Mathematics, University of Washington, Seattle, WA(数学系,华盛顿大学)
A Multi-Level Causal Intervention Framework for Mechanistic Interpretability in Variational Autoencoders
Indian Institute of Technology, Patna(印度理工学院巴特那分校);Rajarshi School of Management Technology, Varanasi(拉贾尔希管理技术学院瓦拉纳西分校);Jaypee Institute of Information Technology, Noida(杰皮信息技术学院诺伊达分校)
Apollo Research(阿波罗研究院);Anthropic(Anthropic公司);Decode Research(解码研究院);Eleuther AI(Eleuther AI公司);FAR AI(FAR AI公司);Google DeepMind(谷歌DeepMind);Leap Laboratories(飞跃实验室);Harvard University(哈佛大学);King’s College London(伦敦国王学院);Imperial College London(帝国理工学院);MATS;MIT(麻省理工学院);METR;Northeastern University(东北大学);Tel Aviv University(特拉维夫大学);Timaeus(提迈奥斯);University of Melbourne(墨尔本大学);Goodfire
Mechanistic Interpretability for Neural TSP Solvers
University of Washington, Michael G. Foster School of Business(华盛顿大学,迈克尔·G·福斯特商学院);University of Washington, Paul G. Allen School of Computer Science & Engineering(华盛顿大学,保罗·G·阿伦计算机科学与工程学院)
Opening the AI black box: program synthesis via mechanistic interpretability
Massachusetts Institute of Technology(麻省理工学院);Institute for Artificial Intelligence and Fundamental Interactions(人工智能与基本相互作用研究所);Harvard University(哈佛大学)
Discovering a Zeta Map Algorithm on Dyck Paths via Mechanistic Interpretability
Department of Mathematics, Temple University, Philadelphia, PA, USA(特拉华大学数学系);Institute for Computer-Aided Reasoning in Mathematics, Carnegie Mellon University, Pittsburgh, PA, USA(计算机辅助数学推理研究所,卡内基梅隆大学);Department of Mathematics, University of Connecticut, Storrs, CT, USA(康乃狄克大学数学系);Korea Institute for Advanced Study, Seoul 02455, Republic of Korea(韩国高等研究院)
Joint data imputation and mechanistic modelling for simulating heart-brain interactions in incomplete datasets
Inria(法国国家信息与自动化技术研究所);Université Côte d’Azur(蔚蓝海岸大学);PhySense(PhySense公司);Department of Information and Communication Technologies(信息与通信技术系);Universitat Pompeu Fabra(庞培法布拉大学)
Prisma: An Open Source Toolkit for Mechanistic Interpretability in Vision and Video
Mila Quebec(蒙特利尔大学);McGill University(麦吉尔大学);Meta;Université de Montréal(蒙特利尔大学);Imperial College London(伦敦帝国理工学院);Fraunhofer Heinrich Hertz Institute(弗劳恩霍夫 Heinrich Hertz 研究所);Technological University Dublin(都柏林技术大学);Apollo Research(Apollo 研究所)
Mechanistic Interpretability Tool for AI Weather Models
Meteorological Institute Munich, Ludwig-Maximilians-Universität, 80333 Munich, Germany(慕尼黑气象研究所,路德维希-马克西米利安大学,德国慕尼黑,80333);Deutsches Zentrum für Luft- und Raumfahrt, Oberpfaffenhofen, Germany(德国航空航天中心,奥伯普法芬霍芬,德国)
Beyond Static Interpretability: Anticipating Post-SFT Mechanisms from Pre-SFT Parameters for Better Tuning
College of Computing and Data Science(计算与数据科学学院);Nanyang Technological University(南洋理工大学);School of Computer Science and Engineering(计算机科学与工程学院);The Chinese University of Hong Kong(香港中文大学)
Mechanistic Interpretability of Antibody Language Models Using SAEs
Department of Statistics, University of Oxford, UK(英国牛津大学统计系);Reticular, San Francisco, USA(美国旧金山Reticular公司);EECS, MIT, Cambridge MA, USA(美国麻省理工学院电子工程与计算机科学系);Leyden Laboratories BV, Leiden, The Netherlands(荷兰莱顿实验室)