线性卷积网络的代数复杂度与神经簇
Algebraic Complexity and Neurovariety of Linear Convolutional Networks
AI总结:
该研究针对带一维滤波器和任意步长的线性卷积网络,提出递归算法生成对应神经流形Zariski闭包的多项式方程,借助度量代数几何发现其优化复临界点数量远超同参数量的全连接线性网络。
摘要:
本文研究具有一维滤波器和任意步长的线性卷积网络。这类网络的神经流形是一个半代数集,由可进行特定因式分解的多项式空间表示。我们提出一种递归算法,生成多项式方程,其公共零点轨迹对应于相应神经流形的Zariski闭包。此外,我们利用度量代数几何的工具,探索训练这类网络的代数复杂度。研究结果表明,此类网络优化过程中所有复临界点的数量等于Segre簇的一般欧氏距离次数。值得注意的是,这一数量远超具有相同参数数量的全连接线性网络训练过程中遇到的临界点数量。
英文摘要:
In this paper, we study linear convolutional networks with one-dimensional filters and arbitrary strides. The neuromanifold of such a network is a semialgebraic set, represented by a space of polynomials admitting specific factorizations. Introducing a recursive algorithm, we generate polynomial equations whose common zero locus corresponds to the Zariski closure of the corresponding neuromanifold. Furthermore, we explore the algebraic complexity of training these networks employing tools from metric algebraic geometry. Our findings reveal that the number of all complex critical points in the optimization of such a network is equal to the generic Euclidean distance degree of a Segre variety. Notably, this count significantly surpasses the number of critical points encountered in the training of a fully connected linear network with the same number of parameters.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。