arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2107.01296cs.LG
收藏

基于子空间聚类的神经网络分析

Subspace Clustering Based Analysis of Neural Networks

University of California Riverside(加州大学河滨分校);Oak Ridge National Laboratory(橡树岭国家实验室)

Uday Singh Saini, Pravallika Devineni, Evangelos E. Papalexakis

摘要:

分析深层神经网络潜在空间的工具为更好地理解它们迈出了一步。在这项工作中,我们激励稀疏子空间聚类(SSC),目的是从一组输入上训练的给定神经网络层的潜在结构中学习亲和图。然后,我们使用来自社区发现的工具来量化输入中存在的结构。这些实验表明,随着我们在网络中的深入,输入往往与同一类的其他输入具有越来越大的亲和力。随后,我们利用矩阵相似性度量来执行亲和图之间的逐层比较。在此过程中,我们首先证明,当将当前正在训练的给定层与其最终状态进行比较时,网络的层越浅,它比更深层收敛得越快。当对整个网络架构进行成对分析时,我们观察到,随着网络规模的增加,它会从每个层与其邻居适度相似的状态重新组织到一个块内的层与其他块中的层具有较高相似性的状态。最后,我们分析了网络最终卷积层的学习亲和图,并演示了输入的局部邻域如何影响网络的分类。

英文摘要:

Tools to analyze the latent space of deep neural networks provide a step towards better understanding them. In this work, we motivate sparse subspace clustering (SSC) with an aim to learn affinity graphs from the latent structure of a given neural network layer trained over a set of inputs. We then use tools from Community Detection to quantify structures present in the input. These experiments reveal that as we go deeper in a network, inputs tend to have an increasing affinity to other inputs of the same class. Subsequently, we utilise matrix similarity measures to perform layer-wise comparisons between affinity graphs. In doing so we first demonstrate that when comparing a given layer currently under training to its final state, the shallower the layer of the network, the quicker it is to converge than the deeper layers. When performing a pairwise analysis of the entire network architecture, we observe that, as the network increases in size, it reorganises from a state where each layer is moderately similar to its neighbours, to a state where layers within a block have high similarity than to layers in other blocks. Finally, we analyze the learned affinity graphs of the final convolutional layer of the network and demonstrate how an input's local neighbourhood affects its classification by the network.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑