arXivDaily arXiv每日学术速递 周一至周五更新
北美退伍军人节(Veterans Day),12 日无更新
arXiv 2103.15341math.NAcs.LGcs.NA
收藏

刚性神经常微分方程

Stiff Neural Ordinary Differential Equations

Massachusetts Institute of Technology(麻省理工学院);Julia Computing Inc.(朱莉娅计算公司);University of Maryland(马里兰大学);Pumas AI(普马斯人工智能)

Suyong Kim, Weiqi Ji, Sili Deng, Yingbo Ma, Christopher Rackauckas

摘要:

神经常微分方程是一种从时间序列数据中学习动态模型的有效方法。本文研究了刚性组系统的神经常微分方程,这类系统通常由化学和生物系统中的化学动力学模型提出。我们首先展示了学习神经ODE在经典的刚性ODE系统的罗伯逊问题的挑战,并提出了技术,以减轻与规模分离的刚性组系统的挑战。然后,我们提出了成功的示范刚性组的罗伯逊问题和空气污染问题。演示表明,使用具有整流激活的深度网络,适当缩放网络输出以及损失函数和稳定梯度计算是实现刚性神经ODE学习的关键技术。学习刚性神经常微分方程的成功开辟了将神经常微分方程用于具有广泛变化时间尺度的应用的可能性,例如能量转换中的化学动力学,环境工程和生命科学。

英文摘要:

Neural Ordinary Differential Equations (ODE) are a promising approach to learn dynamic models from time-series data in science and engineering applications. This work aims at learning Neural ODE for stiff systems, which are usually raised from chemical kinetic modeling in chemical and biological systems. We first show the challenges of learning neural ODE in the classical stiff ODE systems of Robertson's problem and propose techniques to mitigate the challenges associated with scale separations in stiff systems. We then present successful demonstrations in stiff systems of Robertson's problem and an air pollution problem. The demonstrations show that the usage of deep networks with rectified activations, proper scaling of the network outputs as well as loss functions, and stabilized gradient calculations are the key techniques enabling the learning of stiff neural ODE. The success of learning stiff neural ODE opens up possibilities of using neural ODEs in applications with widely varying time-scales, like chemical dynamics in energy conversion, environmental engineering, and the life sciences.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑