多线性压缩感知及其在卷积线性网络中的应用
Multilinear compressive sensing and an application to convolutional linear networks
摘要:
我们研究了一个具有结构的深度线性网络。它采用矩阵$X$的形式,该矩阵是通过将$K$矩阵(称为因子,对应于层的作用)相乘而获得的。通过将固定的线性算子应用于满足约束的参数向量来获得每一层的动作(即因子)。层数不受限制。假设给出了$X$,并且已经估计了因子,则估计因子与$X$的乘积之间的误差(即重建误差)是统计风险或经验风险。在本文中,我们提供了充要条件和充分条件下的网络拓扑结构的稳定性保持。稳定性属性要求定义因子的参数上的误差(即,恢复的参数的稳定性)与重建误差(即,风险)成线性比例。因此,在网络拓扑结构的这些条件下,任何成功的学习任务都会导致稳定定义的特征,从而导致可解释的层/网络。然后,我们证明了任何深度结构化的线性网络都可以转换为一般的多线性问题(使用Segre嵌入)。这是{ \em张量提升}。使用张量提升,我们提供了充要条件的因素(规模重排)的可识别性。最后我们给出了一个称为\NSPlong ~的充要条件(因为与压缩感知框架中通常的零空间性质类似),它保证了稳定性性质成立。我们用一个实际的例子来说明这个理论,其中深度结构化线性网络是一个卷积线性网络。正如预期的那样,条件相当强劲,但并非空洞。可以对网络拓扑进行简单测试,以测试条件是否成立。
英文摘要:
We study a deep linear network endowed with a structure. It takes the form of a matrix $X$ obtained by multiplying $K$ matrices (called factors and corresponding to the action of the layers). The action of each layer (i.e. a factor) is obtained by applying a fixed linear operator to a vector of parameters satisfying a constraint. The number of layers is not limited. Assuming that $X$ is given and factors have been estimated, the error between the product of the estimated factors and $X$ (i.e. the reconstruction error) is either the statistical or the empirical risk. In this paper, we provide necessary and sufficient conditions on the network topology under which a stability property holds. The stability property requires that the error on the parameters defining the factors (i.e. the stability of the recovered parameters) scales linearly with the reconstruction error (i.e. the risk). Therefore, under these conditions on the network topology, any successful learning task leads to stably defined features and therefore interpretable layers/network.In order to do so, we first evaluate how the Segre embedding and its inverse distort distances. Then, we show that any deep structured linear network can be cast as a generic multilinear problem (that uses the Segre embedding). This is the {\em tensorial lifting}. Using the tensorial lifting, we provide necessary and sufficient conditions for the identifiability of the factors (up to a scale rearrangement). We finally provide the necessary and sufficient condition called \NSPlong~(because of the analogy with the usual Null Space Property in the compressed sensing framework) which guarantees that the stability property holds. We illustrate the theory with a practical example where the deep structured linear network is a convolutional linear network. As expected, the conditions are rather strong but not empty. A simple test on the network topology can be implemented to test if the condition holds.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。