arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2302.08183cs.LGcs.AI
收藏

奇异值表示:神经网络的一种新的图形视角

Singular Value Representation: A New Graph Perspective On Neural Networks

École Polytechnique(巴黎综合理工学院);EPFL(洛桑联邦理工学院)

Dan Meller, Nicolas Berkouk

摘要:

我们介绍了奇异值表示(SVR),一种新的方法来表示的内部状态的神经网络使用SVD分解的权重。这种构造产生了一个新的加权图,连接我们称之为谱神经元,对应于经典神经元的特定激活模式。我们推导出一个精确的统计框架来区分全连接和卷积层的谱神经元之间的有意义的连接。为了证明我们的方法对机器学习研究的有用性,我们强调了我们使用SVR的两个发现。首先,我们强调了VGG网络中跨越多个深层的主导连接的出现。其次,我们见证,不依赖于任何输入数据,批量归一化可以诱导深层近核之间的显着连接,导致显着的自发稀疏化现象。

英文摘要:

We introduce the Singular Value Representation (SVR), a new method to represent the internal state of neural networks using SVD factorization of the weights. This construction yields a new weighted graph connecting what we call spectral neurons, that correspond to specific activation patterns of classical neurons. We derive a precise statistical framework to discriminate meaningful connections between spectral neurons for fully connected and convolutional layers. To demonstrate the usefulness of our approach for machine learning research, we highlight two discoveries we made using the SVR. First, we highlight the emergence of a dominant connection in VGG networks that spans multiple deep layers. Second, we witness, without relying on any input data, that batch normalization can induce significant connections between near-kernels of deep layers, leading to a remarkable spontaneous sparsification phenomenon.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑