具有固定偏置配置的深度神经网络
Deep Neural Nets with Fixed Bias Configuration
摘要：
对于任何给定的神经网络架构，权重和偏置的排列都会导致相同的功能网络。这意味着用于“训练”或“学习”网络的优化算法在参数空间中面临着非常大量（即使对于小网络也有数百万）的等效最优解。据我们所知，文献中没有这种观察。为了缩小参数搜索空间，引入了一种新的技术，以固定的偏置向量配置单调增加。这是通过在每一层的偏置向量上增加不等式约束来实现的。针对这些不等式约束，提出了一种基于Moreau-Yosida正则化的算法，并证明了该算法的理论收敛性。所提出的方法的标准三角函数和更具有挑战性的刚性常微分方程中产生的化学反应流的应用程序清楚地说明了所提出的方法的好处。
英文摘要：
For any given neural network architecture a permutation of weights and biases results in the same functional network. This implies that optimization algorithms used to `train' or `learn' the network are faced with a very large number (in the millions even for small networks) of equivalent optimal solutions in the parameter space. To the best of our knowledge, this observation is absent in the literature. In order to narrow down the parameter search space, a novel technique is introduced in order to fix the bias vector configurations to be monotonically increasing. This is achieved by augmenting a typical learning problem with inequality constraints on the bias vectors in each layer. A Moreau-Yosida regularization based algorithm is proposed to handle these inequality constraints and a theoretical convergence of the this algorithm is established. Applications of the proposed approach to standard trigonometric functions and more challenging stiff ordinary differential equations arising in chemically reacting ows clearly illustrate the benefits of the proposed approach.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
