Starting from the Kaya identity, we used a Neural ODE model to predict the evolution of several indicators related to carbon emissions, on a country-level: population, GDP per capita, energy intensity of GDP, carbon intensity of energy. We compared the model with a baseline statistical model - VAR - and obtained good performances. We conclude that this machine-learning approach can be used to produce a wide range of results and give relevant insight to policymakers
Physiome-ODE: A Benchmark for Irregularly Sampled Multivariate Time Series Forecasting Based on Biological ODEs
ISMLL & VWFS DARC University of Hildesheim（ISMLL与VWFS DARC海德堡大学）；University of Hildesheim（海德堡大学）；Institute of Mathematics TU Berlin（柏林技术大学数学研究所）；DARC University of Hildesheim（DARC海德堡大学）
University of Wisconsin-Madison（威斯康星大学麦迪逊分校）；Los Alamos National Laboratory（洛斯阿拉莫斯国家实验室）；Argonne National Laboratory（阿贡国家实验室）
In data-driven modeling of spatiotemporal phenomena careful consideration often needs to be made in capturing the dynamics of the high wavenumbers. This problem becomes especially challenging when the system of interest exhibits shocks or chaotic dynamics. We present a data-driven modeling method that accurately captures shocks and chaotic dynamics by proposing a novel architecture, stabilized neural ordinary differential equation (ODE). In our proposed architecture, we learn the right-hand-side (RHS) of an ODE by adding the outputs of two NN together where one learns a linear term and the oth…
Deep learning has an increasing impact to assist research, allowing, for example, the discovery of novel materials. Until now, however, these artificial intelligence techniques have fallen short of discovering the full differential equation of an experimental physical system. Here we show that a dynamical neural network, trained on a minimal amount of data, can predict the behavior of spintronic devices with high accuracy and an extremely efficient simulation time, compared to the micromagnetic simulations that are usually employed to model them. For this purpose, we re-frame the formalism of …
Florida Atlantic University（佛罗里达大西洋大学）；Harbor Branch Oceanographic Institute, Florida Atlantic University（佛罗里达大西洋大学港口分支海洋研究所）；Georgia Institute of Technology（佐治亚理工学院）；Washington University School of Medicine（华盛顿大学医学院）
Ocean current, fluid mechanics, and many other spatio-temporal physical dynamical systems are essential components of the universe. One key characteristic of such systems is that certain physics laws -- represented as ordinary/partial differential equations (ODEs/PDEs) -- largely dominate the whole process, irrespective of time or location. Physics-informed learning has recently emerged to learn physics for accurate prediction, but they often lack a mechanism to leverage localized spatial and temporal correlation or rely on hard-coded physics parameters. In this paper, we advocate a physics-co…
CTIA-ODE: Continuous-Time Inter-Variable Attention for Long-Term Time Series Forecasting
UGOD, HKUST-GZ（香港科技大学（广州））；School of Software, Yunnan University（云南大学软件学院）；UGOD, The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））
Institute of Informatics, LMU Munich（慕尼黑大学信息学研究所）；Corporate Technology, Siemens AG（西门子股份公司企业技术部）；Technical University of Munich（慕尼黑工业大学）
There has been an increasing interest in inferring future links on temporal knowledge graphs (KG). While links on temporal KGs vary continuously over time, the existing approaches model the temporal KGs in discrete state spaces. To this end, we propose a novel continuum model by extending the idea of neural ordinary differential equations (ODEs) to multi-relational graph convolutional networks. The proposed model preserves the continuous nature of dynamic multi-relational graph data and encodes both temporal and structural information into continuous-time dynamic embeddings. In addition, a nov…
Due to the increasing availability of large-scale observation and simulation datasets, data-driven representations arise as efficient and relevant computation representations of dynamical systems for a wide range of applications, where model-driven models based on ordinary differential equation remain the state-of-the-art approaches. In this work, we investigate neural networks (NN) as physically-sound data-driven representations of such systems. Reinterpreting Runge-Kutta methods as graphical models, we consider a residual NN architecture and introduce bilinear layers to embed non-linearities…
Arizona State University（亚利桑那州立大学）；Oregon State University（俄勒冈州立大学）；Washington University in St. Louis（圣路易斯华盛顿大学）；Yonsei University（延世大学）
Continuous-time dynamics models, such as neural ordinary differential equations, have enabled the modeling of underlying dynamics in time-series data and accurate forecasting. However, parameterization of dynamics using a neural network makes it difficult for humans to identify causal structures in the data. In consequence, this opaqueness hinders the use of these models in the domains where capturing causal relationships carries the same importance as accurate predictions, e.g., tsunami forecasting. In this paper, we address this challenge by proposing a mechanism for mining causal structures…
FRWKV+: Periodic-Aware Adaptive Gating for Frequency-Space Linear Time Series Forecasting
College of Information Science and Engineering, Northeastern University（信息科学与工程学院，东北大学）；Foshan Graduate School of Innovation, Northeastern University（创新研究生学院，东北大学）；National Frontiers Science Center for Industrial Intelligence and Systems Optimization（工业智能与系统优化国家级前沿科学中心）
Black Sigatoka disease severely decreases global banana production, and climate change aggravates the problem by altering fungal species distributions. Due to the heavy financial burden of managing this infectious disease, farmers in developing countries face significant banana crop losses. Though scientists have produced mathematical models of infectious diseases, adapting these models to incorporate climate effects is difficult. We present MR. NODE (Multiple predictoR Neural ODE), a neural network that models the dynamics of black Sigatoka infection learnt directly from data via Neural Ordin…
Research Institute for Infrastructure Paradigm Shift, Yachiyo Engineering Co., Ltd.（基础设施范式转换研究所，八千代工程株式会社）
Recently, flood damage has become a social problem owing to unexperienced weather conditions arising from climate change. An immediate response to heavy rain is important for the mitigation of economic losses and also for rapid recovery. Spatiotemporal precipitation forecasts may enhance the accuracy of dam inflow prediction, more than 6 hours forward for flood damage mitigation. However, the ordinary ConvLSTM has the limitation of predictable range more than 3-timesteps in real-world precipitation forecasting owing to the irreducible bias between target prediction and ground-truth value. This…
PINN-Cast: Exploring the Role of Continuous-Depth NODE in Transformers and Physics Informed Loss as Soft Physical Constraints in Short-term Weather Forecasting
Wavelet-Enhanced Neural ODE and Graph Attention for Interpretable Energy Forecasting
organization= School of Science, Engineering \& Technology, East Delta University , addressline = Abdullah Al Noman Road, Noman Society , city= Chattagram , postcode= 4209 , country= Bangladesh
A comparative study of NeuralODE and Universal ODE approaches to solving Chandrasekhar White Dwarf equation
National Autonomous University of Mexico（墨西哥国立自治大学）；Vizuara AI Labs（Vizuara AI实验室）；Massachusetts Institute of Technology（麻省理工学院）；Purdue University（普渡大学）
University of California, San Diego（加州大学圣地亚哥分校）；SRI International（SRI国际）
Neural Ordinary Differential Equations (NODEs) have proven to be a powerful modeling tool for approximating (interpolation) and forecasting (extrapolation) irregularly sampled time series data. However, their performance degrades substantially when applied to real-world data, especially long-term data with complex behaviors (e.g., long-term trend across years, mid-term seasonality across months, and short-term local variation across days). To address the modeling of such complex data with different behaviors at different frequencies (time spans), we propose a novel progressive learning paradig…
Texas A&M University – Corpus Christi（得克萨斯农工大学科珀斯克里斯蒂分校）；Clarkson University（克拉克森大学）
Classical methods of solving spatiotemporal dynamical systems include statistical approaches such as autoregressive integrated moving average, which assume linear and stationary relationships between systems' previous outputs. Development and implementation of linear methods are relatively simple, but they often do not capture non-linear relationships in the data. Thus, artificial neural networks (ANNs) are receiving attention from researchers in analyzing and forecasting dynamical systems. Recurrent neural networks (RNN), derived from feed-forward ANNs, use internal memory to process variable…
Neural Ordinary Differential Equations (NODEs) are a novel neural architecture, built around initial value problems with learned dynamics which are solved during inference. Thought to be inherently more robust against adversarial perturbations, they were recently shown to be vulnerable to strong adversarial attacks, highlighting the need for formal guarantees. However, despite significant progress in robustness verification for standard feed-forward architectures, the verification of high dimensional NODEs remains an open problem. In this work, we address this challenge and propose GAINS, an a…
University of Colorado, Boulder（科罗拉多大学博尔德分校）；George Mason University（乔治梅森大学）
In this work, we present an approach called Disease Informed Neural Networks (DINNs) that can be employed to effectively predict the spread of infectious diseases. This approach builds on a successful physics informed neural network approaches that have been applied to a variety of applications that can be modeled by linear and non-linear ordinary and partial differential equations. Specifically, we build on the application of PINNs to SIR compartmental models and expand it a scaffolded family of mathematical models describing various infectious diseases. We show how the neural networks are ca…
AdaNODEs: Test Time Adaptation for Time Series Forecasting Using Neural ODEs
The University of Melbourne, Australia（墨尔本大学）；Nokia Bell Labs, UK（诺基亚贝尔实验室）；The University of Auckland, New Zealand（奥克兰大学）；University of Cambridge, UK（剑桥大学）；The University of New South Wales, Australia（新南威尔士大学）；University of Glasgow, UK（格拉斯哥大学）
Neural networks inspired by differential equations have proliferated for the past several years. Neural ordinary differential equations (NODEs) and neural controlled differential equations (NCDEs) are two representative examples of them. In theory, NCDEs provide better representation learning capability for time-series data than NODEs. In particular, it is known that NCDEs are suitable for processing irregular time-series data. Whereas NODEs have been successfully extended after adopting attention, however, it had not been studied yet how to integrate attention into NCDEs. To this end, we pres…
Neural Ordinary Differential Equations for Learning and Extrapolating System Dynamics Across Bifurcations
Biometris, Wageningen University and Research（生物计量研究所，瓦赫宁根大学和研究学院）；Artificial Intelligence Group, Wageningen University and Research（人工智能组，瓦赫宁根大学和研究学院）
Discrete Differential Principle for Continuous Smooth Function Representation
National Key Laboratory of Science & Technology on Multi-Spectral Information Processing（国家多谱信息处理科技重点实验室）；College of artificial intelligence and automation（人工智能与自动化学院）；Huazhong University of Science and Technology（华中科技大学）
Graph Neural ODE Digital Twins for Control-Oriented Reactor Thermal-Hydraulic Forecasting Under Partial Observability
Department of Nuclear Engineering, Texas A&M University, College Station, TX 77843, USA（德克萨斯A&M大学核工程系，学院站，TX 77843，美国）；Argonne National Laboratory, Nuclear Science and Engineering Division, USA（阿贡国家实验室，核科学与工程部，美国）
