Model-free inference of unseen attractors: Reconstructing phase space features from a single noisy trajectory using reservoir computing
Instituto de Física Interdisciplinar y Sistemas Complejos, IFISC (CSIC-UIB)（跨学科物理与复杂系统研究所 (IFISC (CSIC-UIB))）；The University of Tokyo（东京大学）；The Ohio State University（俄亥俄州立大学）；ResCon Technologies, LLC（ResCon Technologies公司）
Reservoir computers are powerful tools for chaotic time series prediction. They can be trained to approximate phase space flows and can thus both predict future values to a high accuracy, as well as reconstruct the general properties of a chaotic attractor without requiring a model. In this work, we show that the ability to learn the dynamics of a complex system can be extended to systems with co-existing attractors, here a 4-dimensional extension of the well-known Lorenz chaotic system. We demonstrate that a reservoir computer can infer entirely unexplored parts of the phase space: a properly trained reservoir computer can predict the existence of attractors that were never approached during training and therefore are labelled as unseen. We provide examples where attractor inference is achieved after training solely on a single noisy trajectory.
Attractor-merging Crises and Intermittency in Reservoir Computing
Graduate School of Information Science and Technology, University of Tokyo, Bunkyo-ku, Tokyo 113-0033, Japan（信息科学与技术研究生学校，东京大学，东京都，东京113-0033，日本）；International Research Center for Neurointelligence, University of Tokyo Institutes for Advanced Study, University of Tokyo, Bunkyo-ku, Tokyo 113-0033, Japan（神经智能国际研究中心，东京大学先进研究所，东京大学，东京都，东京113-0033，日本）
The Ohio State University（俄亥俄州立大学）；ResCon Technologies LLC（ResCon技术有限责任公司）；Instituto de Física Interdisciplinar y Sistemas Complejos, IFISC (CSIC-UIB)（跨学科物理与复杂系统研究所（IFISC，西班牙国家研究委员会-巴利阿里群岛大学））；The University of Tokyo（东京大学）
Reservoir computing is a machine learning approach that can generate a surrogate model of a dynamical system. It can learn the underlying dynamical system using fewer trainable parameters and hence smaller training data sets than competing approaches. Recently, a simpler formulation, known as next-generation reservoir computing, removes many algorithm metaparameters and identifies a well-performing traditional reservoir computer, thus simplifying training even further. Here, we study a particularly challenging problem of learning a dynamical system that has both disparate time scales and multi…
University College Cork（科克大学）；Raytheon Technologies Research Center Ireland（雷神技术爱尔兰研究中心）
Multifunctionality is a well observed phenomenological feature of biological neural networks and considered to be of fundamental importance to the survival of certain species over time. These multifunctional neural networks are capable of performing more than one task without changing any network connections. In this paper we investigate how this neurological idiosyncrasy can be achieved in an artificial setting with a modern machine learning paradigm known as `Reservoir Computing'. A training technique is designed to enable a Reservoir Computer to perform tasks of a multifunctional nature. We…
Designing Chaotic Attractors: A Semi-supervised Approach
Graduate School of Information Science and Technology, University of Tokyo（东京大学情报理工学系研究科）；International Research Center for Neurointelligence, University of Tokyo Institutes for Advanced Study, University of Tokyo（东京大学高等研究院国际神经智能研究中心）
Multifunctional physical reservoir computing in soft tensegrity robots
School of Information Science and Technology, The University of Tokyo, Tokyo, Japan（信息科学与技术学系，东京大学，东京，日本）；Next Generation Artificial Intelligence Research Center (AI Center), The University of Tokyo, Tokyo, Japan（下一代人工智能研究中心（AI中心），东京大学，东京，日本）
Prediction of Nonlinear Oscillations in a Jumping Quarter-Car Model Using Reservoir Computing
Tokyo University of Agriculture and Technology（东京农工大学）；Amity University Haryana - Gurugram（Amity大学哈里亚纳邦-古鲁格拉姆校区）；Nagoya Institute of Technology（名古屋工业大学）；Central University of Rajasthan（中央拉贾斯坦大学）
Institute of Mathematics and Computer Science (ICMC) University of São Paulo (USP)（圣保罗大学数学与计算机科学研究所）
Dynamical systems has a variety of applications for the new information generated during the time. Many phenomenons like physical, chemical or social are not static, then an analysis over the time is necessary. In this work, an experimental analysis of parameters of the model Echo State Network is performed and the influence of the kind of Complex Network is explored to understand the influence on the performance. The experiments are performed using the Rossler attractor.
Machine learning recently proved efficient in learning differential equations and dynamical systems from data. However, the data is commonly assumed to originate from a single never-changing system. In contrast, when modeling real-world dynamical processes, the data distribution often shifts due to changes in the underlying system dynamics. Continual learning of these processes aims to rapidly adapt to abrupt system changes without forgetting previous dynamical regimes. This work proposes an approach to continual learning based on reservoir computing, a state-of-the-art method for training rec…
Embedding Theory of Reservoir Computing and Reducing Reservoir Network Using Time Delays
Soochow University（苏州大学）；Fudan University（复旦大学）；Potsdam Institute for Climate Impact Research (PIK)（波茨坦气候影响研究所（PIK））；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）
Denoising and Reconstruction of Nonlinear Dynamics using Truncated Reservoir Computing
Centre for Audio, Acoustics and Vibration (CAAV), School of Mechanical and Mechatronic Engineering, University of Technology Sydney, Ultimo, NSW 2007, Australia（音频、声学与振动中心（CAAV），机械与机电工程学院，悉尼技术大学，Ultimo，新南威尔士州2007，澳大利亚）
Minimal Deterministic Echo State Networks Outperform Random Reservoirs in Learning Chaotic Dynamics
Center for Scalable Data Analytics；Artificial Intelligence (ScaDS.AI), Leipzig University, Leipzig, Germany；Institute for Earth System Science \& Remote Sensing, Leipzig University, Leipzig, Germany
Next-Generation Reservoir Computing for Dynamical Inference
Centre for Mathematical Science, Lund University（数学科学中心，隆德大学）；IMFUFA, Department of Science and Environment, Roskilde University（IMFUFA，科学与环境系，罗斯基尔德大学）
Reconstructing dynamics from sparse observations with no training on target system
School of Electrical, Computer and Energy Engineering, Arizona State University（亚利桑那州立大学电气、计算机与能源工程学院）；Doctor of Physical Therapy Program, Tufts University School of Medicine（塔夫茨大学医学院物理治疗博士项目）；Department of Physics, Arizona State University（亚利桑那州立大学物理系）
Reduced-order dynamical models play a central role in developing our understanding of predictability of climate irrespective of whether we are dealing with the actual climate system or surrogate climate-models. In this context, the Linear-Inverse-Modeling (LIM) approach, by capturing a few essential interactions between dynamical components of the full system, has proven valuable in providing insights into predictability of the full system. We demonstrate that Reservoir Computing (RC), a form of learning suitable for systems with chaotic dynamics, provides an alternative nonlinear approach tha…
Exploring the origins of switching dynamics in a multifunctional reservoir computer
School of Mathematical Sciences, University College Cork（科克大学学院数学科学学院）；INFANT Research Centre, University College Cork（科克大学学院INFANT研究中心）；Potsdam Institute for Climate Impact Research（波茨坦气候影响研究所）
University College Cork（科克大学学院）；Ludwig-Maximilians University Munich（慕尼黑路德维希-马克西米利安大学）；Collins Aerospace（柯林斯航空航天公司）；Deutsches Zentrum für Luft-und Raumfahrt（德国航空航天中心）
Multifunctional neural networks are capable of performing more than one task without changing any network connections. In this paper we explore the performance of a continuous-time, leaky-integrator, and next-generation `reservoir computer' (RC), when trained on tasks which test the limits of multifunctionality. In the first task we train each RC to reconstruct a coexistence of chaotic attractors from different dynamical systems. By moving the data describing these attractors closer together, we find that the extent to which each RC can reconstruct both attractors diminishes as they begin to o…
Arizona State University（亚利桑那州立大学）；Shaanxi Normal University（陕西师范大学）；University of Aberdeen（阿伯丁大学）
To predict a critical transition due to parameter drift without relying on model is an outstanding problem in nonlinear dynamics and applied fields. A closely related problem is to predict whether the system is already in or if the system will be in a transient state preceding its collapse. We develop a model free, machine learning based solution to both problems by exploiting reservoir computing to incorporate a parameter input channel. We demonstrate that, when the machine is trained in the normal functioning regime with a chaotic attractor (i.e., before the critical transition), the transit…
The Ohio State University（俄亥俄州立大学）；Potomac Research LLC（波托马克研究有限责任公司）
We explore the hyperparameter space of reservoir computers used for forecasting of the chaotic Lorenz '63 attractor with Bayesian optimization. We use a new measure of reservoir performance, designed to emphasize learning the global climate of the forecasted system rather than short-term prediction. We find that optimizing over this measure more quickly excludes reservoirs that fail to reproduce the climate. The results of optimization are surprising: the optimized parameters often specify a reservoir network with very low connectivity. Inspired by this observation, we explore reservoir design…
Nanyang Technological University（南洋理工大学）；School of Materials Science and Engineering（材料科学与工程学院）
We introduce a self-consistent deep-learning framework which, for a noisy deterministic time series, provides unsupervised filtering, state-space reconstruction, identification of the underlying differential equations and forecasting. Without a priori information on the signal, we embed the time series in a state space, where deterministic structures, i.e. attractors, are revealed. Under the assumption that the evolution of solution trajectories is described by an unknown dynamical system, we filter out stochastic outliers. The embedding function, the solution trajectories and the dynamical sy…
School of Physics and Information Technology, Shaanxi Normal University（陕西师范大学物理与信息技术学院）
Reconstructing the KAM dynamics diagram of Hamiltonian system from the time series of a limited number of parameters is an outstanding question in nonlinear science, especially when the Hamiltonian governing the system dynamics are unknown. Here, we demonstrate that this question can be addressed by the machine learning approach knowing as reservoir computer (RC). Specifically, we show that without prior knowledge about the Hamilton's equations of motion, the trained RC is able to not only predict the short-term evolution of the system state, but also replicate the long-term ergodic properties…
A scalable generative model for dynamical system reconstruction from neuroimaging data
Central Institute of Mental Health (CIMH)（中央精神卫生研究所（CIMH））；Medical Faculty Mannheim, Heidelberg University（海德堡大学曼海姆医学院）；Johannes Kepler University（约翰内斯·开普勒大学）；Interdisciplinary Center for Scientific Computing, Heidelberg University（海德堡大学跨学科科学计算中心）；Faculty of Physics and Astronomy, Heidelberg University（海德堡大学物理与天文学学院）；Hector Institute for AI in Psychiatry（赫克托精神病学人工智能研究所）；Faculty of Mathematics and Computer Science, Heidelberg University（海德堡大学数学与计算机科学学院）
