Almost Sure Asymptotic Freeness of Neural Network Jacobian with Orthogonal Weights
英文摘要：
A well-conditioned Jacobian spectrum has a vital role in preventing exploding or vanishing gradients and speeding up learning of deep neural networks. Free probability theory helps us to understand and handle the Jacobian spectrum. We rigorously show almost sure asymptotic freeness of layer-wise Jacobians of deep neural networks as the wide limit. In particular, we treat the case that weights are initialized as Haar distributed orthogonal matrices.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
