arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1812.01892math.NAcs.NA
收藏

微分方程解导数的自动微分与连续敏感性分析比较

A Comparison of Automatic Differentiation and Continuous Sensitivity Analysis for Derivatives of Differential Equation Solutions

Julia Computing(朱莉娅计算);Pumas-AI;New York University(纽约大学);Massachusetts Institute of Technology(麻省理工学院)

Yingbo Ma, Vaibhav Dixit, Mike Innes, Xingjian Guo, Christopher Rackauckas

AI总结:

本文比较了自动微分与连续敏感性分析在微分方程解导数计算中的性能,发现对于小规模ODE系统,自动微分方法更高效,而连续伴随法在规模增大后更具优势,同时展示了自动微分在处理微分代数方程、时滞微分方程等复杂系统时的优势。

摘要:

微分方程解的导数常用于参数估计、神经微分方程拟合和模型诊断。然而,由于众多选择和潜在方法的笛卡尔积,从业者难以确定哪种方法在特定应用中最为有效。本文研究了通过自动微分实现的离散局部敏感性分析(DSAAD)与连续伴随敏感性分析的性能特征。使用非刚性和刚性的生物和药代动力学模型,包括偏微分方程的离散化,以量化敏感性分析方法的性能。我们的基准测试显示,在小规模ODE系统(约<100参数+ODEs)中,前向模式DSAAD比反向模式和连续前向/伴随敏感性分析更高效。连续伴随方法的可扩展性在超过此规模范围后比离散伴随和前向方法更高效。这些比较研究展示了在连续伴随方法中内存使用与性能之间的权衡,应考虑在选择技术时。同时,机器学习文献中数值不稳定的回解技术被证明不适合大多数科学模型。伴随方法的性能与反向模式自动微分方法密切相关,tape-based自动微分方法在非线性偏微分方程上比静态自动微分技术慢两个数量级。这些结果还展示了DSAAD在微分代数方程、时滞微分方程和混合微分方程系统中的适用性,展示了DSAAD方法在实现上的便利性。

英文摘要:

Derivatives of differential equation solutions are commonly for parameter estimation, fitting neural differential equations, and as model diagnostics. However, with a litany of choices and a Cartesian product of potential methods, it can be difficult for practitioners to understand which method is likely to be the most effective on their particular application. In this manuscript we investigate the performance characteristics of Discrete Local Sensitivity Analysis implemented via Automatic Differentiation (DSAAD) against continuous adjoint sensitivity analysis. Non-stiff and stiff biological and pharmacometric models, including a PDE discretization, are used to quantify the performance of sensitivity analysis methods. Our benchmarks show that on small systems of ODEs (approximately $<100$ parameters+ODEs), forward-mode DSAAD is more efficient than both reverse-mode and continuous forward/adjoint sensitivity analysis. The scalability of continuous adjoint methods is shown to be more efficient than discrete adjoints and forward methods after crossing this size range. These comparative studies demonstrate a trade-off between memory usage and performance in the continuous adjoint methods that should be considered when choosing the technique, while numerically unstable backsolve techniques from the machine learning literature are demonstrated as unsuitable for most scientific models. The performance of adjoint methods is shown to be heavily tied to the reverse-mode AD method, with tape-based AD methods shown to be 2 orders of magnitude slower on nonlinear partial differential equations than static AD techniques. These results also demonstrate the applicability of DSAAD to differential-algebraic equations, delay differential equations, and hybrid differential equation systems, showcasing an ease of implementation advantage for DSAAD approaches.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑