Automated interpretability systems aim to reduce the need for human labor and scale analysis to increasingly large models and diverse tasks. Recent efforts toward this goal leverage large language models (LLMs) at increasing levels of autonomy, ranging from fixed one-shot workflows to fully autonomous interpretability agents. This shift creates a corresponding need to scale evaluation approaches to keep pace with both the volume and complexity of generated explanations. We investigate this challenge in the context of automated circuit analysis -- explaining the roles of model components when performing specific tasks. To this end, we build an agentic system in which a research agent iteratively designs experiments and refines hypotheses. When evaluated against human expert explanations across six circuit analysis tasks in the literature, the system appears competitive. However, closer examination reveals several pitfalls of replication-based evaluation: human expert explanations can be subjective or incomplete, outcome-based comparisons obscure the research process, and LLM-based systems may reproduce published findings via memorization or informed guessing. To address some of these pitfalls, we propose an unsupervised intrinsic evaluation based on the functional interchangeability of model components. Our work demonstrates fundamental challenges in evaluating complex automated interpretability systems and reveals key limitations of replication-based evaluation.
Deutsche Welle（德国之声）；Universitat Politecnica de Valencia（瓦伦西亚理工大学）；ABC News（美国广播公司新闻部）；Bonn-Aachen International Center for Information Technology (b-it), University of Bonn（波恩大学波恩-亚琛国际信息技术中心）；The Lamarr Institute for Machine Learning and Artificial Intelligence（拉马尔机器学习与人工智能研究所）
Microsoft Research Asia（微软亚洲研究院）；Beijing Normal University（北京师范大学）；Universitat Politècnica de València（瓦伦西亚理工大学）；ValgrAI；The Psychometrics Centre, University of Cambridge（剑桥大学心理测量中心）；Cambridge Judge Business School（剑桥大学嘉治商学院）
Evaluating Supervised Machine Learning Models: Principles, Pitfalls, and Metric Selection
Jiangsu Key Laboratory of Big Data Security and Intelligent Processing, Nanjing University of Posts and Telecommunications（江苏大数据安全与智能处理重点实验室，南京邮电大学）；University of Brighton（布莱顿大学）；University of Lancashire（兰开夏大学）；School of Cyber Science and Engineering, Southeast University（东南大学信息科学与工程学院）
Evaluating Post-hoc Interpretability with Intrinsic Interpretability
University of Coimbra（科英布拉大学）；IPO-Porto Research Centre（波尔图肿瘤学研究所研究中心）；University of Applied Sciences Western Switzerland (HES-SO Valais)（瑞士西部应用科学大学（瓦莱州高等专业学院））
University of Maryland College Park（马里兰大学帕克分校）；University of Virginia（弗吉尼亚大学）；Cornell University（康奈尔大学）；Columbia University（哥伦比亚大学）；University of California Los Angeles（加州大学洛杉矶分校）
Interpretability Benchmark for Evaluating Spatial Misalignment of Prototypical Parts Explanations
Jagiellonian University（雅盖隆大学）；Doctoral School of Exact and Natural Sciences, Jagiellonian University（雅盖隆大学精确与自然科学博士学院）；Faculty of Management and Social Communication, Jagiellonian University（雅盖隆大学管理与社会传播学院）；IDEAS NCBR
Evaluating the Interpretability of Sparse Autoencoders with Concept Annotations
The Berlin Institute for the Foundations of Learning and Data (BIFOLD)（柏林学习与数据基础研究所）；Technische Universität Berlin（柏林工业大学）；INRAE（法国国家农业、食品与环境研究院）；Inria, EVERGREEN（法国国家信息与自动化研究所，EVERGREEN）；UMR TETIS, Univ Montpellier（UMR TETIS，蒙彼利埃大学）
A Unified Framework for Evaluating and Enhancing the Transparency of Explainable AI Methods via Perturbation-Gradient Consensus Attribution
Department of Computer Science, American International University-Bangladesh（美国国际大学-孟加拉国计算机科学系）；Thomas Lord Department of Computer Science, Viterbi School of Engineering, University of Southern California（南加州大学维特比工程学院托马斯·洛德计算机科学系）；Department of Computer Science and Engineering, Techno International New Town（Techno International New Town计算机科学与工程系）
Developing Future Human-Centered Smart Cities: Critical Analysis of Smart City Security, Interpretability, and Ethical Challenges
Hamad Bin Khalifa University（哈马德·本·哈利法大学）；The Hashemite University（哈希姆大学）；Pak-Austria Fachhochschule: Institute of Applied Sciences and Technology（巴基斯坦-奥地利应用科学与技术学院）；Qatar University（卡塔尔大学）；Information Technology University（信息技术大学）
