基于加性高斯噪声变换的多层网络信息的加性
Additivity of Information in Multilayer Networks via Additive Gaussian Noise Transforms
摘要：
多层（或深度）网络是一种强大的概率模型，它基于线性变换的多个阶段，然后是非线性（可能是随机）函数。通常，线性变换由矩阵定义，非线性函数由信息通道定义。这些模型已经获得了很大的普及，由于他们的能力，复杂的概率关系，在各种各样的推理问题的特点。本文的贡献是一种新的方法，用于分析的基本限制的统计推断的设置模型是已知的。我们的方法的有效性可以在许多设置中建立，并被证明更普遍地持有。贯穿始终的一个关键假设是，矩阵是从正交不变分布中随机抽取的。我们的方法产生明确的公式1）的互信息; 2）最小均方误差（MMSE）; 3）的存在和位置的某些相变相对于问题的参数;和4）的稳定点的状态演化的近似消息传递算法。当应用到具有多元高斯通道的模型的特殊情况时，我们的方法是严格的，并且与随机矩阵的自由概率理论有密切的联系。当应用到非高斯信道的一般情况下，我们的方法提供了一个简单的替代副本方法从统计物理。一个关键的观察结果是，当在某个变换域中观察时，模型中各个分量（即矩阵和通道）的组合效应是相加的。
英文摘要：
Multilayer (or deep) networks are powerful probabilistic models based on multiple stages of a linear transform followed by a non-linear (possibly random) function. In general, the linear transforms are defined by matrices and the non-linear functions are defined by information channels. These models have gained great popularity due to their ability to characterize complex probabilistic relationships arising in a wide variety of inference problems. The contribution of this paper is a new method for analyzing the fundamental limits of statistical inference in settings where the model is known. The validity of our method can be established in a number of settings and is conjectured to hold more generally. A key assumption made throughout is that the matrices are drawn randomly from orthogonally invariant distributions. Our method yields explicit formulas for 1) the mutual information; 2) the minimum mean-squared error (MMSE); 3) the existence and locations of certain phase-transitions with respect to the problem parameters; and 4) the stationary points for the state evolution of approximate message passing algorithms. When applied to the special case of models with multivariate Gaussian channels our method is rigorous and has close connections to free probability theory for random matrices. When applied to the general case of non-Gaussian channels, our method provides a simple alternative to the replica method from statistical physics. A key observation is that the combined effects of the individual components in the model (namely the matrices and the channels) are additive when viewed in a certain transform domain.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
