深度线性网络中对角矩阵补全的熵选择失败
Failure of entropic selection for diagonal matrix completion in deep linear networks
浏览论文内容
中文总结 AI 辅助
本文证明在深度线性网络的对角矩阵补全中，有限温度纤维熵无法提供均衡选择，因为自由能无下界且几乎所有满秩轨迹发散。
中文摘要 AI 辅助
我们研究了在可逆实$d\times d$矩阵空间上与深度为$N$的深度线性网络相关的自由能梯度流。损失函数仅观察对角线，\\[ E_d(W)=\frac12\sum_{i=1}^d(W_{ii}-1)^2, \\] 正则化项是由Menon和Yu计算的平衡分解纤维的玻尔兹曼熵。这是Menon提出的矩阵补全问题的一个自然的高维版本，用于测试熵是否在非紧致的最小值族中进行选择。对于对角补全，我们证明对于每个宽度$d\geq2$、深度$N>2$和逆温度$\beta>0$，自由能无下界，并且恰好有$2^d$个满秩临界点，每个对角符号区域一个。每个临界点都是对角的且双曲的。其不稳定维数为$\binom d2$，稳定维数为$d(d+1)/2$。我们还证明了一个满秩轨迹不能收敛到有限秩亏矩阵。因此，几乎每个满秩初始条件都有一个无界的前向轨道。因此，有限温度纤维熵不能为对角补全提供均衡选择原则。
英文摘要
We study the free-energy gradient flow associated with the depth-$N$ deep linear network on the space of invertible real $d\times d$ matrices. The loss observes only the diagonal, \[ E_d(W)=\frac12\sum_{i=1}^d(W_{ii}-1)^2, \] and the regularizer is the Boltzmann entropy of the balanced factorization fiber computed by Menon and Yu. This is a natural higher-dimensional version of a matrix-completion problem posed by Menon as a test of whether entropy selects among a noncompact family of minimizers. For diagonal completion, every width $d\geq2$, depth $N>2$, and inverse temperature $β>0$, we prove that the free energy is unbounded below and has exactly $2^d$ full-rank critical points, one in each diagonal sign chamber. Every critical point is diagonal and hyperbolic. Its unstable dimension is $\binom d2$ and its stable dimension is $d(d+1)/2$. We also prove that a full-rank trajectory cannot converge to a finite rank-deficient matrix. Consequently, almost every full-rank initial condition has an unbounded forward orbit. Thus finite-temperature fiber entropy does not provide an equilibrium selection principle for diagonal completion.
