arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2303.02536cs.AI
收藏

在可解释因果变量与分布式神经表示之间寻找对齐

Finding Alignments Between Interpretable Causal Variables and Distributed Neural Representations

Stanford University(斯坦福大学)

Atticus Geiger, Zhengxuan Wu, Christopher Potts, Thomas Icard, Noah D. Goodman

AI总结:

本文提出分布式对齐搜索(DAS),用梯度下降替代暴力搜索,并允许神经元在分布式表示中承担多重角色,从而更有效地发现训练神经网络中的概念结构。

摘要:

因果抽象是可解释人工智能中一个有前景的理论框架,它定义了可解释的高层因果模型何时是低层深度学习系统的忠实简化。然而,现有的因果抽象方法存在两个主要局限:它们需要对高层模型与低层模型之间的对齐进行暴力搜索,并且它们预设高层模型中的变量会与低层模型中互不相交的神经元集合对齐。在本文中,我们提出了分布式对齐搜索(DAS),以克服这些局限。在 DAS 中,我们使用梯度下降而非暴力搜索来寻找高层模型与低层模型之间的对齐,并且我们允许单个神经元通过分析非标准基下的表示——即分布式表示——来扮演多个不同的角色。我们的实验表明,DAS 能够发现先前方法遗漏的内部结构。总体而言,DAS 消除了进行因果抽象分析的既有障碍,并使我们能够在训练好的神经网络中发现概念结构。

英文摘要:

Causal abstraction is a promising theoretical framework for explainable artificial intelligence that defines when an interpretable high-level causal model is a faithful simplification of a low-level deep learning system. However, existing causal abstraction methods have two major limitations: they require a brute-force search over alignments between the high-level model and the low-level one, and they presuppose that variables in the high-level model will align with disjoint sets of neurons in the low-level one. In this paper, we present distributed alignment search (DAS), which overcomes these limitations. In DAS, we find the alignment between high-level and low-level models using gradient descent rather than conducting a brute-force search, and we allow individual neurons to play multiple distinct roles by analyzing representations in non-standard bases-distributed representations. Our experiments show that DAS can discover internal structure that prior approaches miss. Overall, DAS removes previous obstacles to conducting causal abstraction analyses and allows us to find conceptual structure in trained neural nets.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑