arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2007.03609math.NAcs.NA
收藏

结构探测神经网络收缩

Structure Probing Neural Network Deflation

Yiqi Gu, Chunmei Wang, Haizhao Yang

摘要:

深度学习是求解非线性微分方程的有力工具,但通常情况下,由于随机梯度下降的隐式正则化,只能找到与最平坦局部最小值对应的解。本文提出了一种基于网络的结构探测收缩方法,使深度学习能够识别非线性物理模型中普遍存在且重要的多个解。首先,我们引入通缩运营商建立与已知的解决方案,使已知的解决方案不再局部极小的优化能量景观。其次,为了便于收敛到期望的局部极小,提出了一种结构探测技术,以获得接近期望的局部极小的初始猜测。结合本文精心设计的神经网络结构,新的正则化优化可以有效地收敛到新解。由于深度学习的无网格性质,所提出的方法能够解决具有多个解的复杂域上的高维问题,而现有方法仅关注一维或二维规则域,并且在运算次数上更昂贵。数值实验也表明,所提出的方法可以找到更多的解决方案比现有的方法。

英文摘要:

Deep learning is a powerful tool for solving nonlinear differential equations, but usually, only the solution corresponding to the flattest local minimizer can be found due to the implicit regularization of stochastic gradient descent. This paper proposes a network-based structure probing deflation method to make deep learning capable of identifying multiple solutions that are ubiquitous and important in nonlinear physical models. First, we introduce deflation operators built with known solutions to make known solutions no longer local minimizers of the optimization energy landscape. Second, to facilitate the convergence to the desired local minimizer, a structure probing technique is proposed to obtain an initial guess close to the desired local minimizer. Together with neural network structures carefully designed in this paper, the new regularized optimization can converge to new solutions efficiently. Due to the mesh-free nature of deep learning, the proposed method is capable of solving high-dimensional problems on complicated domains with multiple solutions, while existing methods focus on merely one or two-dimensional regular domains and are more expensive in operation counts. Numerical experiments also demonstrate that the proposed method could find more solutions than exiting methods.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑