arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2111.06206cs.LGcs.AIcs.CV
收藏

定义与量化深度神经网络中稀疏概念的出现

Defining and Quantifying the Emergence of Sparse Concepts in DNNs

Shanghai Jiao Tong University(上海交通大学)

Jie Ren, Mingjie Li, Qirui Chen, Huiqi Deng, Quanshi Zhang

AI总结:

本文研究了DNN中的概念涌现现象,发现其推理得分可解耦为稀疏符号因果图中的少数交互概念,并证明该因果图能忠实解释DNN输出,还可简化为And-Or图且保持较高解释精度。

摘要:

本文旨在阐明训练好的DNN中的概念涌现现象。具体而言,我们发现DNN的推理得分可被解耦为少数交互概念的影响。这些概念可被理解为稀疏符号因果图中的因果模式,用以解释DNN。用此因果图解释DNN的忠实性在理论上是有保证的,因为我们证明了该因果图能很好地模拟DNN在指数级数量的不同掩码样本上的输出。此外,该因果图可被进一步简化并重写为And-Or图(AOG),且不会损失太多解释精度。

英文摘要:

This paper aims to illustrate the concept-emerging phenomenon in a trained DNN. Specifically, we find that the inference score of a DNN can be disentangled into the effects of a few interactive concepts. These concepts can be understood as causal patterns in a sparse, symbolic causal graph, which explains the DNN. The faithfulness of using such a causal graph to explain the DNN is theoretically guaranteed, because we prove that the causal graph can well mimic the DNN's outputs on an exponential number of different masked samples. Besides, such a causal graph can be further simplified and re-written as an And-Or graph (AOG), without losing much explanation accuracy.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑