arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2211.12031math.NAcs.NA
收藏

有限神经元方法的一种逐神经元子空间校正方法

A neuron-wise subspace correction method for the finite neuron method

Department of Mathematics, Pennsylvania State University(宾夕法尼亚州立大学数学系)

Jongho Park, Jinchao Xu, Xiaofeng Xu

AI总结:

本文针对有限神经元方法求解偏微分方程时训练精度不足的问题,提出逐神经元并行子空间校正方法NPSC,通过分离优化线性层与各神经元并引入最优预条件子和超线性收敛算法,在函数逼近和PDE实验中优于传统梯度方法。

摘要:

在本文中,我们提出了一种新的算法,称为逐神经元并行子空间校正方法(NPSC),用于有限神经元方法,该方法利用神经网络函数逼近偏微分方程(PDEs)的数值解。尽管在将神经网络应用于数值求解偏微分方程方面已有极其广泛的研究活动,但即便对于一维问题,仍然严重缺乏能够达到足够精度的有效训练算法。基于近期关于线性层谱性质以及单神经元问题景观分析的结果,我们发展了一类特殊的子空间校正方法,该方法分别优化线性层和非线性层中的每个神经元。针对一维问题,我们给出了一个最优预条件子,用于解决线性层的病态条件问题,使得线性层的训练迭代次数相对于神经元数量保持均匀。在每个单神经元问题中,通过一个超线性收敛算法找到一个良好的局部极小值,从而避开平坦能量区域。在函数逼近问题和偏微分方程上的数值实验表明,所提出的方法比其他基于梯度的方法具有更好的性能。

英文摘要:

In this paper, we propose a novel algorithm called Neuron-wise Parallel Subspace Correction Method (NPSC) for the finite neuron method that approximates numerical solutions of partial differential equations (PDEs) using neural network functions. Despite extremely extensive research activities in applying neural networks for numerical PDEs, there is still a serious lack of effective training algorithms that can achieve adequate accuracy, even for one-dimensional problems. Based on recent results on the spectral properties of linear layers and landscape analysis for single neuron problems, we develop a special type of subspace correction method that optimizes the linear layer and each neuron in the nonlinear layer separately. An optimal preconditioner that resolves the ill-conditioning of the linear layer is presented for one-dimensional problems, so that the linear layer is trained in a uniform number of iterations with respect to the number of neurons. In each single neuron problem, a good local minimum that avoids flat energy regions is found by a superlinearly convergent algorithm. Numerical experiments on function approximation problems and PDEs demonstrate better performance of the proposed method than other gradient-based methods.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑