arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2210.00319cs.LG
收藏

PathFinder:发现深度神经网络中的决策路径

PathFinder: Discovering Decision Pathways in Deep Neural Networks

Bloomberg L.P.(彭博有限合伙企业);Özyeğin University(厄兹耶金大学)

Ozan İrsoy, Ethem Alpaydın

摘要:

可解释性正成为深度神经网络越来越重要的主题。虽然卷积层中的操作更容易理解,但在全连接层中处理变得不透明。我们工作的基本思想是,每个实例在流经各层时,都会在隐藏层中产生不同的激活模式,在我们的路径方法中,我们将每个隐藏层的激活向量聚类,然后查看连续层中的聚类如何在激活从输入层流向输出时相互连接。我们发现,同一个类的实例在层上遵循少量的聚类序列,我们称之为“决策路径”。这些路径解释了分类决策通常是如何做出的,也帮助我们确定遵循不寻常路径的离群值。我们还建议使用桑基图可视化这样的途径。我们验证了我们的方法与实验上的两个前馈网络训练MNIST和CELEB数据集,和一个循环网络训练PenDigits。

英文摘要:

Explainability is becoming an increasingly important topic for deep neural networks. Though the operation in convolutional layers is easier to understand, processing becomes opaque in fully-connected layers. The basic idea in our work is that each instance, as it flows through the layers, causes a different activation pattern in the hidden layers and in our Paths methodology, we cluster these activation vectors for each hidden layer and then see how the clusters in successive layers connect to one another as activation flows from the input layer to the output. We find that instances of the same class follow a small number of cluster sequences over the layers, which we name ``decision paths." Such paths explain how classification decisions are typically made, and also help us determine outliers that follow unusual paths. We also propose using the Sankey diagram to visualize such pathways. We validate our method with experiments on two feed-forward networks trained on MNIST and CELEB data sets, and one recurrent network trained on PenDigits.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑