因果抽象是可解释人工智能中一个有前景的理论框架,它定义了可解释的高层因果模型何时是低层深度学习系统的忠实简化。然而,现有的因果抽象方法存在两个主要局限:它们需要对高层模型与低层模型之间的对齐进行暴力搜索,并且它们预设高层模型中的变量会与低层模型中互不相交的神经元集合对齐。在本文中,我们提出了分布式对齐搜索(DAS),以克服这些局限。在 DAS 中,我们使用梯度下降而非暴力搜索来寻找高层模型与低层模型之间的对齐,并且我们允许单个神经元通过分析非标准基下的表示——即分布式表示——来扮演多个不同的角色。我们的实验表明,DAS 能够发现先前方法遗漏的内部结构。总体而言,DAS 消除了进行因果抽象分析的既有障碍,并使我们能够在训练好的神经网络中发现概念结构。
英文摘要:
Causal abstraction is a promising theoretical framework for explainable artificial intelligence that defines when an interpretable high-level causal model is a faithful simplification of a low-level deep learning system. However, existing causal abstraction methods have two major limitations: they require a brute-force search over alignments between the high-level model and the low-level one, and they presuppose that variables in the high-level model will align with disjoint sets of neurons in the low-level one. In this paper, we present distributed alignment search (DAS), which overcomes these limitations. In DAS, we find the alignment between high-level and low-level models using gradient descent rather than conducting a brute-force search, and we allow individual neurons to play multiple distinct roles by analyzing representations in non-standard bases-distributed representations. Our experiments show that DAS can discover internal structure that prior approaches miss. Overall, DAS removes previous obstacles to conducting causal abstraction analyses and allows us to find conceptual structure in trained neural nets.
Learning Interpretable Concepts: Unifying Causal Representation Learning and Foundation Models
Carnegie Mellon University(卡内基梅隆大学);Max Planck Institute for Intelligent Systems(马克斯·普朗克智能系统研究所);Tübingen AI Center(蒂宾根人工智能中心);University of Chicago(芝加哥大学);ELLIS Institute Tübingen(ELLIS蒂宾根研究所)
Towards Causally Interpretable Wi-Fi CSI-Based Human Activity Recognition with Discrete Latent Compression and LTL Rule Extraction
6 Ohio University, USA.\;4 U.S. Army DEVCOM Army Research Laboratory, USA.\;8 Cardiff University, UK.\;9 University of Southampton, UK.\;10 Imperial College London, UK.\
Graph Similarity Computation via Interpretable Neural Node Alignment
School of Computer Science, Hangzhou Dianzi University(杭州电子科技大学计算机学院);School of Data Science, Lingnan University(岭南大学数据科学学院);School of Science and Technology, Hong Kong Metropolitan University(香港都会大学科技学院)
CoSToM:Causal-oriented Steering for Intrinsic Theory-of-Mind Alignment in Large Language Models
National Engineering Research Center for Big Data Technology and System, Services Computing Technology and System Lab, Cluster and Grid Computing Lab, Huazhong University of Science and Technology(华中科技大学国家大数据技术与系统工程技术研究中心、服务计算技术与系统实验室、集群与网格计算实验室);Singapore Management University(新加坡管理大学)
SphUnc: Hyperspherical Uncertainty Decomposition and Causal Identification via Information Geometry
University of Macau(澳门大学);Fudan University(复旦大学);Hangzhou Dianzi University(杭州电子科技大学);Renmin University of China(中国人民大学);Hanyang University(翰阳大学);Tsinghua University(清华大学);Zhejiang University(浙江大学);University of Liverpool(利物浦大学);Southeast University(东南大学);Central South University(中南大学)
Towards Interpretable Geo-localization: a Concept-Aware Global Image-GPS Alignment Framework
Peking University(北京大学);Harbin Institute of Technology(哈尔滨工业大学);The Hong Kong University of Science and Technology(香港科技大学);Harbin Institute of Technology (Weihai)(哈尔滨工业大学(威海))
From Concept Alignment to Causal Grounding: An Intervention Test of Chain-of-Thought Faithfulness
German Research Center for Artificial Intelligence (DFKI)(德国人工智能研究中心(DFKI));Saarland Informatics Campus(萨尔兰信息学园区);Centre for European Research in Trusted AI (CERTAIN)(欧洲可信人工智能研究中心(CERTAIN));Technische Universität Berlin(柏林工业大学);IBM Research(IBM研究院);University of Copenhagen(哥本哈根大学);University of Groningen(格罗宁根大学)
Interpretable Neural ODEs for Gene Regulatory Network Discovery under Perturbations
Department of Computer Science, Columbia University, New York, U.S.(哥伦比亚大学计算机科学系);Department of Industrial Engineering and Operations Research, Columbia University, New York, U.S.(哥伦比亚大学工业工程与运筹学系);Department of Applied Mathematics and Applied Physics, Columbia University, New York, U.S.(哥伦比亚大学应用数学与应用物理系);New York Genome Center, New York, U.S.(纽约基因组中心);Irving Institute of Cancer Dynamics, New York, U.S.(伊万·罗伯特癌症动力学研究所);Institute of Computational Biology, Helmholtz Munich, Munich, Germany(海德堡医学院计算生物学研究所);Department of Mathematics, Technische Universität München, Munich, Germany(慕尼黑技术大学数学系)