神经网络的体-边界分解
Bulk-boundary decomposition of neural networks
AI总结:
提出体-边界分解框架,将神经网络训练动力学分解为数据无关的体项和数据相关的边界项,揭示深层网络的局部齐次结构并推导能量连续性方程。
摘要:
我们提出体-边界分解作为理解深度神经网络训练动力学的新框架。从随机梯度下降公式出发,我们证明拉格朗日量可以重组为数据无关的体项和数据相关的边界项。体项捕捉由网络架构和激活函数设定的内在动力学,而边界项反映来自输入和输出层训练样本的随机相互作用。这种分解揭示了深层网络背后的局部和齐次结构。作为局部性和齐次性的物理结果,我们推导了深度神经网络内的能量连续性方程。
英文摘要:
We present the bulk--boundary decomposition as a new framework for understanding the training dynamics of deep neural networks. Starting from the stochastic gradient descent formulation, we show that the Lagrangian can be reorganized into a data-independent bulk term and a data-dependent boundary term. The bulk captures the intrinsic dynamics set by network architecture and activation functions, while the boundary reflects stochastic interactions from training samples at the input and output layers. This decomposition exposes the local and homogeneous structure underlying deep networks. As a physical consequence of locality and homogeneity, we derive the energy continuity equation within a deep neural network.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。