Taylor-Lagrange神经常微分方程：神经常微分方程的快速训练和评估
Taylor-Lagrange Neural Ordinary Differential Equations: Toward Fast Training and Evaluation of Neural ODEs
摘要：
神经常微分方程（NODE）--使用神经网络对微分方程进行参数化--在从数据中学习未知连续时间动态系统的模型方面显示出巨大的前景。然而，节点的每个前向评估都需要对用于捕获系统动态的神经网络进行数值积分，这使得它们的训练过于昂贵。现有的工作依赖于现成的自适应步长数值积分方案，这往往需要大量的评估的基础动力学网络，以获得足够的精度训练。相比之下，我们加快了评估和培训的NODE提出了一个数据驱动的方法，他们的数值积分。所提出的泰勒-拉格朗日节点（TL-NODE）使用固定阶泰勒展开进行数值积分，同时还学习估计展开的近似误差。因此，所提出的方法实现了相同的精度，自适应步长计划，而只采用低阶泰勒展开，从而大大降低了计算成本，需要集成的NODE。一套数值实验，包括建模动态系统，图像分类，和密度估计，表明TL-NODE可以训练超过一个数量级的速度比国家的最先进的方法，没有任何损失的性能。
英文摘要：
Neural ordinary differential equations (NODEs) -- parametrizations of differential equations using neural networks -- have shown tremendous promise in learning models of unknown continuous-time dynamical systems from data. However, every forward evaluation of a NODE requires numerical integration of the neural network used to capture the system dynamics, making their training prohibitively expensive. Existing works rely on off-the-shelf adaptive step-size numerical integration schemes, which often require an excessive number of evaluations of the underlying dynamics network to obtain sufficient accuracy for training. By contrast, we accelerate the evaluation and the training of NODEs by proposing a data-driven approach to their numerical integration. The proposed Taylor-Lagrange NODEs (TL-NODEs) use a fixed-order Taylor expansion for numerical integration, while also learning to estimate the expansion's approximation error. As a result, the proposed approach achieves the same accuracy as adaptive step-size schemes while employing only low-order Taylor expansions, thus greatly reducing the computational cost necessary to integrate the NODE. A suite of numerical experiments, including modeling dynamical systems, image classification, and density estimation, demonstrate that TL-NODEs can be trained more than an order of magnitude faster than state-of-the-art approaches, without any loss in performance.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
