arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2012.07244cs.LG
收藏

贝叶斯神经常微分方程

Bayesian Neural Ordinary Differential Equations

Massachusetts Institute of Technology（麻省理工学院）；Julia Computing Inc.（朱莉娅计算公司）；Pumas AI（普马斯人工智能公司）；University of New South Wales at Canberra（堪培拉新南威尔士大学）；National Autonomous University of Mexico（墨西哥国立自治大学）；Pramati Technologies（普拉马蒂科技公司）；University of Maryland Baltimore（马里兰大学巴尔的摩分校）

Raj Dandekar, Karen Chung, Vaibhav Dixit, Mohamed Tarek, Aslan Garcia-Valadez, Krishna Vishal Vemula, Chris Rackauckas

备注：
16 pages, 10 figures, 3 tables; added new inference methods, substantially improved MNIST accuracy, revised author affiliations
阅读 PDF
arXiv 页面 ↗HTML 原文 ↗

摘要：

最近，神经常微分方程已经成为一个强大的框架，用于建模物理模拟，而无需显式定义控制系统的ODE，而是通过机器学习来学习它们。然而，问题是：贝叶斯学习框架能否与神经ODE集成，以稳健地量化神经ODE权重的不确定性？“仍然没有答案为了解决这个问题，我们主要评估以下几类推理方法：（a）无U形转弯MCMC采样器（NUTS），（b）随机梯度哈密顿蒙特卡罗（SGHMC）和（c）随机朗之万梯度下降（SGLD）。我们使用GPU加速，在经典物理系统以及MNIST等标准机器学习数据集上成功集成了神经ODE与上述贝叶斯推理框架。在MNIST数据集上，我们在10，000张图像的测试集合上实现了98.5%的后验样本准确率。随后，我们第一次展示了变分推理与归一化流和神经ODE的成功集成，从而产生了一个强大的贝叶斯神经ODE对象。最后，考虑一个捕食者-食饵模型和一个流行病系统，利用泛常微分方程证明了部分描述动力系统模型规格的概率辨识。总之，这给出了一个科学的机器学习工具的概率估计的认知不确定性。

英文摘要：

Recently, Neural Ordinary Differential Equations has emerged as a powerful framework for modeling physical simulations without explicitly defining the ODEs governing the system, but instead learning them via machine learning. However, the question: "Can Bayesian learning frameworks be integrated with Neural ODE's to robustly quantify the uncertainty in the weights of a Neural ODE?" remains unanswered. In an effort to address this question, we primarily evaluate the following categories of inference methods: (a) The No-U-Turn MCMC sampler (NUTS), (b) Stochastic Gradient Hamiltonian Monte Carlo (SGHMC) and (c) Stochastic Langevin Gradient Descent (SGLD). We demonstrate the successful integration of Neural ODEs with the above Bayesian inference frameworks on classical physical systems, as well as on standard machine learning datasets like MNIST, using GPU acceleration. On the MNIST dataset, we achieve a posterior sample accuracy of 98.5% on the test ensemble of 10,000 images. Subsequently, for the first time, we demonstrate the successful integration of variational inference with normalizing flows and Neural ODEs, leading to a powerful Bayesian Neural ODE object. Finally, considering a predator-prey model and an epidemiological system, we demonstrate the probabilistic identification of model specification in partially-described dynamical systems using universal ordinary differential equations. Together, this gives a scientific machine learning tool for probabilistic estimation of epistemic uncertainties.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

↑