从Algebraic Geometry看多层线性神经网络
The Landscape of Multi-Layer Linear Neural Network From the Perspective of Algebraic Geometry
摘要：
对神经网络非凸性的清晰认识是一个复杂的不完全问题。本文研究了非线性网络的简化形式--线性（剩余）网络的景观。将梯度方程视为多项式方程，利用代数几何工具在复数域上求解，所得解可分解为不同的不可约复几何对象。然后提出了三个假设，分别涉及如何计算每个不可约几何对象上的损失、临界点的损失具有一定的范围以及每个不可约几何对象的维数与严格鞍条件的关系。最后，应用数值代数几何方法验证了这三个假设的合理性，进一步阐明了线性网络的景观和剩余连接的作用。
英文摘要：
The clear understanding of the non-convex landscape of neural network is a complex incomplete problem. This paper studies the landscape of linear (residual) network, the simplified version of the nonlinear network. By treating the gradient equations as polynomial equations, we use algebraic geometry tools to solve it over the complex number field, the attained solution can be decomposed into different irreducible complex geometry objects. Then three hypotheses are proposed, involving how to calculate the loss on each irreducible geometry object, the losses of critical points have a certain range and the relationship between the dimension of each irreducible geometry object and strict saddle condition. Finally, numerical algebraic geometry is applied to verify the rationality of these three hypotheses which further clarify the landscape of linear network and the role of residual connection.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
