It has been observed that residual networks can be viewed as the explicit Euler discretization of an Ordinary Differential Equation (ODE). This observation motivated the introduction of so-called Neural ODEs, which allow more general discretization schemes with adaptive time stepping. Here, we propose ANODEV2, which is an extension of this approach that also allows evolution of the neural network parameters, in a coupled ODE-based formulation. The Neural ODE method introduced earlier is in fact a special case of this new more general framework. We present the formulation of ANODEV2, derive optimality conditions, and implement a coupled reaction-diffusion-advection version of this framework in PyTorch. We present empirical results using several different configurations of ANODEV2, testing them on multiple models on CIFAR-10. We report results showing that this coupled ODE-based framework is indeed trainable, and that it achieves higher accuracy, as compared to the baseline models as well as the recently-proposed Neural ODE approach.
Mitigating Global Feature Collapse in Graph ODEs via Hysteretic Topology Evolution
Doctoral Program of Computer Science, University of Helsinki, Helsinki, Finland（赫尔辛基大学计算机科学博士项目）；Research Program in Systems Oncology, Faculty of Medicine, University of Helsinki, Helsinki, Finland（赫尔辛基大学医学系系统肿瘤研究计划）
Samsung Advanced Institute of Technology（三星先进技术研究院）；Samsung Electronics（三星电子）；Korea Advanced Institute of Science and Technology (KAIST)（韩国科学技术院（KAIST））
Predicting dynamical system evolution with residual neural networks
Keldysh Institute of Applied Mathematics of the Russian Academy of Sciences（俄罗斯科学院克尔德什应用数学研究所）；Skolkovo Institute of Science and Technology（斯科尔科沃科学技术学院）；Marchuk Institute of Numerical Mathematics of the Russian Academy of Sciences（俄罗斯科学院马尔丘克数值数学研究所）
Google Brain（谷歌大脑）；Columbia University（哥伦比亚大学）；DeepMind（深度思维）；Stanford University（斯坦福大学）；University of Cambridge（剑桥大学）；Massachusetts Institute of Technology（麻省理工学院）；The Alan Turing Institute（艾伦·图灵研究所）
Time-varying Interaction Graph ODE for Dynamic Graph Representation Learning
Key Laboratory of Computational Intelligence and Chinese Information Processing of Ministry of Education（教育部计算智能与中文信息处理重点实验室）；School of Computer and Information Technology（计算机与信息学院）；Shanxi University（山西大学）；Department of System Engineering and Engineering Management（系统工程与工程管理系）；Key Laboratory of Complex Systems and Data Science of Ministry of Education（教育部复杂系统与数据科学重点实验室）；Complex Systems Research Center（复杂系统研究中心）
