arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2006.04791cs.LGcond-mat.dis-nncs.NEhep-thstat.ML
收藏

深度神经网络的复杂性和深度特征表示的其他特征

Complexity for deep neural networks and other characteristics of deep feature representations

Jagiellonian University(雅盖隆大学)

Romuald A. Janik, Przemek Witaszczyk

摘要:

我们定义了一个复杂性的概念,它量化了一个神经网络的计算的非线性,以及特征表示的有效维度的补充措施。我们研究了各种数据集的训练网络的这些观测值,并在训练过程中探索它们的动态,特别是幂律缩放。这些可观测量可以用双重方式来理解,即揭示数据集本身作为尺度或深度的函数的隐藏内部结构。所提出的复杂性概念的熵特征应该允许将分析模式从神经科学和统计物理学转移到人工神经网络的领域。所引入的观测量可以应用于生物神经元系统的分析而不作任何改变。

英文摘要:

We define a notion of complexity, which quantifies the nonlinearity of the computation of a neural network, as well as a complementary measure of the effective dimension of feature representations. We investigate these observables both for trained networks for various datasets as well as explore their dynamics during training, uncovering in particular power law scaling. These observables can be understood in a dual way as uncovering hidden internal structure of the datasets themselves as a function of scale or depth. The entropic character of the proposed notion of complexity should allow to transfer modes of analysis from neuroscience and statistical physics to the domain of artificial neural networks. The introduced observables can be applied without any change to the analysis of biological neuronal systems.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑