arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2202.04925cond-mat.dis-nnstat.ML
收藏

将神经网络分解为相关函数的映射

Decomposing neural networks as mappings of correlation functions

Kirsten Fischer, Alexandre René, Christian Keup, Moritz Layer, David Dahmen, Moritz Helias

AI总结:

本文研究深度前馈网络实现的概率分布映射,将其描述为分布的迭代变换,揭示每层非线性在不同阶相关函数间传递信息的机制,为分类提供定量且可解释的视角。

摘要:

理解深度神经网络中信息处理的功能原理仍然是一个挑战,特别是对于具有已训练(即非随机)权重的网络。为了解决这个问题,我们研究了由深度前馈网络实现的概率分布之间的映射。我们将此映射描述为分布的迭代变换,其中每层的非线性在相关函数的不同阶之间传递信息。这使我们能够识别数据中的关键统计量,以及神经网络可以使用的不同信息表示。通过应用于XOR任务和MNIST,我们表明高达二阶的相关性主要捕获内部层中的信息处理,而输入层也从数据中提取高阶相关性。该分析为分类提供了定量且可解释的视角。

英文摘要:

Understanding the functional principles of information processing in deep neural networks continues to be a challenge, in particular for networks with trained and thus non-random weights. To address this issue, we study the mapping between probability distributions implemented by a deep feed-forward network. We characterize this mapping as an iterated transformation of distributions, where the non-linearity in each layer transfers information between different orders of correlation functions. This allows us to identify essential statistics in the data, as well as different information representations that can be used by neural networks. Applied to an XOR task and to MNIST, we show that correlations up to second order predominantly capture the information processing in the internal layers, while the input layer also extracts higher-order correlations from the data. This analysis provides a quantitative and explainable perspective on classification.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑