神经网络训练的Port-Hamilton方法
Port-Hamiltonian Approach to Neural Network Training
摘要:
神经网络是离散的实体:细分为离散层,并通过权重进行参数化,权重通过差分方程进行迭代优化。最近的工作提出了网络的层输出不再量化,但常微分方程(ODE)的解决方案;然而,这些网络仍然通过离散方法(例如梯度下降)进行优化。在本文中,我们探索了一个不同的方向:即,我们提出了一个新的框架,学习中的参数本身是解决方案的常微分方程。通过将优化过程视为端口-哈密顿系统的演化,我们可以确保收敛到目标函数的最小值。数值实验表明了所提方法的有效性和可行性。
英文摘要:
Neural networks are discrete entities: subdivided into discrete layers and parametrized by weights which are iteratively optimized via difference equations. Recent work proposes networks with layer outputs which are no longer quantized but are solutions of an ordinary differential equation (ODE); however, these networks are still optimized via discrete methods (e.g. gradient descent). In this paper, we explore a different direction: namely, we propose a novel framework for learning in which the parameters themselves are solutions of ODEs. By viewing the optimization process as the evolution of a port-Hamiltonian system, we can ensure convergence to a minimum of the objective function. Numerical experiments have been performed to show the validity and effectiveness of the proposed methods.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。