Large Language Models (LLMs) have demonstrated unprecedented capabilities across various natural language processing tasks. Their ability to process and generate viable text and code has made them ubiquitous in many fields, while their deployment as knowledge bases and "reasoning" tools remains an area of ongoing research. In geography, a growing body of literature has been focusing on evaluating LLMs' geographical knowledge and their ability to perform spatial reasoning. However, very little is still known about the internal functioning of these models, especially about how they process geographical information.
In this chapter, we establish a novel framework for the study of geospatial mechanistic interpretability - using spatial analysis to reverse engineer how LLMs handle geographical information. Our aim is to advance our understanding of the internal representations that these complex models generate while processing geographical information - what one might call "how LLMs think about geographic information" if such phrasing was not an undue anthropomorphism.
We first outline the use of probing in revealing internal structures within LLMs. We then introduce the field of mechanistic interpretability, discussing the superposition hypothesis and the role of sparse autoencoders in disentangling polysemantic internal representations of LLMs into more interpretable, monosemantic features. In our experiments, we use spatial autocorrelation to show how features obtained for placenames display spatial patterns related to their geographic location and can thus be interpreted geospatially, providing insights into how these models process geographical information. We conclude by discussing how our framework can help shape the study and use of foundation models in geography.
When the Coffee Feature Activates on Coffins: An Analysis of Feature Extraction and Steering for Mechanistic Interpretability
Department of Philosophy of Nature and Technology(自然哲学与技术系);Munich School of Philosophy(慕尼黑哲学学院);Division of the Humanities and Social Sciences(人文与社会科学系);California Institute of Technology(加州理工学院)
A Multi-Level Causal Intervention Framework for Mechanistic Interpretability in Variational Autoencoders
Indian Institute of Technology, Patna(印度理工学院巴特那分校);Rajarshi School of Management Technology, Varanasi(拉贾尔希管理技术学院瓦拉纳西分校);Jaypee Institute of Information Technology, Noida(杰皮信息技术学院诺伊达分校)
Apollo Research(阿波罗研究院);Anthropic(Anthropic公司);Decode Research(解码研究院);Eleuther AI(Eleuther AI公司);FAR AI(FAR AI公司);Google DeepMind(谷歌DeepMind);Leap Laboratories(飞跃实验室);Harvard University(哈佛大学);King’s College London(伦敦国王学院);Imperial College London(帝国理工学院);MATS;MIT(麻省理工学院);METR;Northeastern University(东北大学);Tel Aviv University(特拉维夫大学);Timaeus(提迈奥斯);University of Melbourne(墨尔本大学);Goodfire
Mechanistic Interpretability for Neural TSP Solvers
University of Washington, Michael G. Foster School of Business(华盛顿大学,迈克尔·G·福斯特商学院);University of Washington, Paul G. Allen School of Computer Science & Engineering(华盛顿大学,保罗·G·阿伦计算机科学与工程学院)
Opening the AI black box: program synthesis via mechanistic interpretability
Massachusetts Institute of Technology(麻省理工学院);Institute for Artificial Intelligence and Fundamental Interactions(人工智能与基本相互作用研究所);Harvard University(哈佛大学)
Discovering a Zeta Map Algorithm on Dyck Paths via Mechanistic Interpretability
Department of Mathematics, Temple University, Philadelphia, PA, USA(特拉华大学数学系);Institute for Computer-Aided Reasoning in Mathematics, Carnegie Mellon University, Pittsburgh, PA, USA(计算机辅助数学推理研究所,卡内基梅隆大学);Department of Mathematics, University of Connecticut, Storrs, CT, USA(康乃狄克大学数学系);Korea Institute for Advanced Study, Seoul 02455, Republic of Korea(韩国高等研究院)
MINAR: Mechanistic Interpretability for Neural Algorithmic Reasoning
Pacific Northwest National Laboratory, Richland, WA(太平洋西北国家实验室);Halıcıoğlu Data Science Institute, University of California, San Diego, San Diego, CA(哈利奇奥格鲁数据科学研究所,加州大学圣地亚哥分校);Department of Computer and Information Science, University of Pennsylvania, Pennsylvaina, PA(计算机与信息科学系,宾夕法尼亚大学);Department of Mathematics, University of Washington, Seattle, WA(数学系,华盛顿大学)
Bridging Mechanistic Interpretability and Prompt Engineering with Gradient Ascent for Interpretable Persona Control
Department of Computer Science(计算机科学系);Indian Institute of Technology (ISM), Dhanbad(印度理工学院(ISM),丹巴德);National University of Singapore(新加坡国立大学);CIFAR Fellow(CIFAR研究员)
Prisma: An Open Source Toolkit for Mechanistic Interpretability in Vision and Video
Mila Quebec(蒙特利尔大学);McGill University(麦吉尔大学);Meta;Université de Montréal(蒙特利尔大学);Imperial College London(伦敦帝国理工学院);Fraunhofer Heinrich Hertz Institute(弗劳恩霍夫 Heinrich Hertz 研究所);Technological University Dublin(都柏林技术大学);Apollo Research(Apollo 研究所)