Neural Ordinary Differential Equations (NODEs) are a new class of models that transform data continuously through infinite-depth architectures. The continuous nature of NODEs has made them particularly suitable for learning the dynamics of complex physical systems. While previous work has mostly been focused on first order ODEs, the dynamics of many systems, especially in classical physics, are governed by second order laws. In this work, we consider Second Order Neural ODEs (SONODEs). We show how the adjoint sensitivity method can be extended to SONODEs and prove that the optimisation of a first order coupled ODE is equivalent and computationally more efficient. Furthermore, we extend the theoretical understanding of the broader class of Augmented NODEs (ANODEs) by showing they can also learn higher order dynamics with a minimal number of augmented dimensions, but at the cost of interpretability. This indicates that the advantages of ANODEs go beyond the extra space offered by the augmented dimensions, as originally thought. Finally, we compare SONODEs and ANODEs on synthetic and real dynamical systems and demonstrate that the inductive biases of the former generally result in faster training and better performance.
CAMEO: Curiosity Augmented Metropolis for Exploratory Optimal Policies
Ecole Polytechnique(巴黎综合理工学院);IRT SystemX(法国系统X技术研究所);Universidad Carlos III de Madrid(马德里卡洛斯三世大学);Universidad Rey Juan Carlos(胡安卡洛斯国王大学);IP-Paris(巴黎理工学院)
Towards Complex Dynamic Physics System Simulation with Graph Neural ODEs
Department of Chemical and Biological Engineering, Faculty of Engineering, Monash University(莫纳什大学化学与生物工程系);Department of Data Science and Artificial Intelligence, Faculty of IT, Monash University(莫纳什大学数据科学与人工智能系);School of Information and Communication Technology, Griffith University(格里菲斯大学信息与通信技术学院);Department of Computer Science, University of Illinois at Chicago(伊利诺伊大学芝加哥分校计算机科学系)
NORi: An ML-Augmented Ocean Boundary Layer Parameterization
Department of Earth, Atmospheric and Planetary Sciences, Massachusetts Institute of Technology(麻省理工学院地球、大气与行星科学系);Center for Computational Science and Engineering, Massachusetts Institute of Technology(麻省理工学院计算科学与工程中心);Department of Physics, Imperial College London(伦敦帝国学院物理系);atdepth;Aeolus Labs(Aeolus实验室);Department of Environment, Land and Infrastructure Engineering, Politecnico di Torino(托里诺理工学院环境、土地与基础设施工程系)