转移神经进化可以可追溯地解决你的微分方程吗?
Can Transfer Neuroevolution Tractably Solve Your Differential Equations?
摘要:
本文介绍了求解微分方程的神经进化算法。通过优化一个深度神经网络,其损失函数由微分方程的残差项定义,得到了该解。最近的研究集中在通过随机梯度下降(SGD)变体来学习这种物理信息神经网络,但由于优化挑战,它们面临着获得准确解的困难。在解微分方程的背景下,我们面临的问题是找到网络的全局最优参数,而不是关注样本外泛化。SGD沿着单一梯度方向搜索,容易陷入局部最优,因此它可能不是这里的最佳方法。相比之下,神经进化对不同的解决方案进行并行探索,目标是绕过局部最优解。它可以通过更好的优化神经网络找到更准确的解决方案。然而,神经进化可能是缓慢的,在实践中提出了易处理性问题。考虑到这一点,本文提出了一种新的和计算效率高的转移神经进化算法。我们的方法是能够利用相关的经验先验知识解决一个新的问题时,与适应,以防止负迁移的风险。该算法应用于各种微分方程的经验证明,转移神经进化确实可以实现更好的精度和更快的收敛速度比SGD。因此,实验结果确立了转移神经进化作为一个值得注意的方法来解决微分方程,一个从来没有被研究过。我们的工作扩展了现有算法的资源,用于优化物理信息神经网络。
英文摘要:
This paper introduces neuroevolution for solving differential equations. The solution is obtained through optimizing a deep neural network whose loss function is defined by the residual terms from the differential equations. Recent studies have focused on learning such physics-informed neural networks through stochastic gradient descent (SGD) variants, yet they face the difficulty of obtaining an accurate solution due to optimization challenges. In the context of solving differential equations, we are faced with the problem of finding globally optimum parameters of the network, instead of being concerned with out-of-sample generalization. SGD, which searches along a single gradient direction, is prone to become trapped in local optima, so it may not be the best approach here. In contrast, neuroevolution carries out a parallel exploration of diverse solutions with the goal of circumventing local optima. It could potentially find more accurate solutions with better optimized neural networks. However, neuroevolution can be slow, raising tractability issues in practice. With that in mind, a novel and computationally efficient transfer neuroevolution algorithm is proposed in this paper. Our method is capable of exploiting relevant experiential priors when solving a new problem, with adaptation to protect against the risk of negative transfer. The algorithm is applied on a variety of differential equations to empirically demonstrate that transfer neuroevolution can indeed achieve better accuracy and faster convergence than SGD. The experimental outcomes thus establish transfer neuroevolution as a noteworthy approach for solving differential equations, one that has never been studied in the past. Our work expands the resource of available algorithms for optimizing physics-informed neural networks.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。