Panel data involving longitudinal measurements of the same set of participants taken over multiple time points is common in studies to understand childhood development and disease modeling. Deep hybrid models that marry the predictive power of neural networks with physical simulators such as differential equations, are starting to drive advances in such applications. The task of modeling not just the observations but the hidden dynamics that are captured by the measurements poses interesting statistical/computational questions. We propose a probabilistic model called ME-NODE to incorporate (fixed + random) mixed effects for analyzing such panel data. We show that our model can be derived using smooth approximations of SDEs provided by the Wong-Zakai theorem. We then derive Evidence Based Lower Bounds for ME-NODE, and develop (efficient) training algorithms using MC based sampling methods and numerical ODE solvers. We demonstrate ME-NODE's utility on tasks spanning the spectrum from simulations and toy data to real longitudinal 3D imaging data from an Alzheimer's disease (AD) study, and study its performance in terms of accuracy of reconstruction for interpolation, uncertainty estimates and personalized prediction.
Neural ODE Transformers: Analyzing Internal Dynamics and Adaptive Fine-tuning
Korea University（高丽大学）；Johns Hopkins University（约翰斯·霍普金斯大学）；Texas A&M University-Commerce（德州农工大学 Commerce 分校）；Qualcomm AI Research（高通人工智能研究部）；Stanford（斯坦福大学）；KAIST（韩国科学技术院）；Qualcomm Vietnam Company Limited（高通越南有限公司）
We propose a novel second-order optimization framework for training the emerging deep continuous-time models, specifically the Neural Ordinary Differential Equations (Neural ODEs). Since their training already involves expensive gradient computation by solving a backward ODE, deriving efficient second-order methods becomes highly nontrivial. Nevertheless, inspired by the recent Optimal Control (OC) interpretation of training deep networks, we show that a specific continuous-time OC methodology, called Differential Programming, can be adopted to derive backward ODEs for higher-order derivatives…
University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
In this paper, we introduce a unified framework for analyzing a large family of Q-learning algorithms, based on switching system perspectives and ODE-based stochastic approximation. We show that the nonlinear ODE models associated with these Q-learning algorithms can be formulated as switched linear systems, and analyze their asymptotic stability by leveraging existing switching system theories. Our approach provides the first O.D.E. analysis of the asymptotic convergence of various Q-learning algorithms, including asynchronous Q-learning and averaging Q-learning. We also extend the approach t…
Causal Discovery from Heteroscedastic Stochastic Dynamical Systems under Imperfect Physical Models
Department of Mechanical & Industrial Engineering（机械与工业工程系）；Northeastern University（东北大学）；Department of Industrial Engineering and Management Sciences（工业工程与管理科学系）；Department of Mechanical Engineering（机械工程系）；Northwestern University（西北大学）
Integrating GNN and Neural ODEs for Estimating Non-Reciprocal Two-Body Interactions in Mixed-Species Collective Motion
Institute of Industrial Science, The University of Tokyo（东京大学工业科学研究所）；Universal Biology Institute, The University of Tokyo（东京大学环球生物研究所）；Graduate School of Arts and Sciences, The University of Tokyo（东京大学文理研究生院）
