Auto-Encoded Reservoir Computing for Turbulence Learning
Delft University of Technology（代尔夫特理工大学）；Technical University of Munich（慕尼黑工业大学）；University of Cambridge（剑桥大学）；The Alan Turing Institute（艾伦·图灵研究所）；Imperial College London（伦敦帝国学院）
Nguyen Anh Khoa Doan, Wolfgang Polifke, Luca Magri
We present an Auto-Encoded Reservoir-Computing (AE-RC) approach to learn the dynamics of a 2D turbulent flow. The AE-RC consists of an Autoencoder, which discovers an efficient manifold representation of the flow state, and an Echo State Network, which learns the time evolution of the flow in the manifold. The AE-RC is able to both learn the time-accurate dynamics of the flow and predict its first-order statistical moments. The AE-RC approach opens up new possibilities for the spatio-temporal prediction of turbulence with machine learning.
Photonic reservoir computing is a promising candidate for low-energy computing at high bandwidths. Despite recent successes, there are bounds to what one can achieve simply by making photonic reservoirs larger. Therefore, a switch from single-reservoir computing to multi-reservoir and even deep physical reservoir computing is desirable. Given that backpropagation can not be used directly to train multi-reservoir systems in our targeted setting, we propose an alternative approach that still uses its power to derive intermediate targets. In this work we report our findings on a conducted experim…
The Ohio State University（俄亥俄州立大学）；ResCon Technologies, LLC（ResCon Technologies公司）；Clarkson University（克拉克森大学）
Reservoir computing is a best-in-class machine learning algorithm for processing information generated by dynamical systems using observed time-series data. Importantly, it requires very small training data sets, uses linear optimization, and thus requires minimal computing resources. However, the algorithm uses randomly sampled matrices to define the underlying recurrent neural network and has a multitude of metaparameters that must be optimized. Recent results demonstrate the equivalence of reservoir computing to nonlinear vector autoregression, which requires no random matrices, fewer metap…
Identifying Systems with Symmetries using Equivariant Autoregressive Reservoir Computers
Department of Statistics and Research, CNBS（统计与研究系，CNBS）；Department of Applied Mathematics, UNAH（应用数学系，UNAH）；Department of Chemical Engineering, Norwegian University of Science and Technology（化学工程系，挪威科学技术大学）
John A. Paulson School of Engineering and Applied Sciences（约翰·A·保尔森工程与应用科学学院）；Harvard University（哈佛大学）
Reservoir computers (RCs) are among the fastest to train of all neural networks, especially when they are compared to other recurrent neural networks. RC has this advantage while still handling sequential data exceptionally well. However, RC adoption has lagged other neural network models because of the model's sensitivity to its hyper-parameters (HPs). A modern unified software package that automatically tunes these parameters is missing from the literature. Manually tuning these numbers is very difficult, and the cost of traditional grid search methods grows exponentially with the number of …
Saitama University（埼玉大学）；University of Tokyo（东京大学）
Photonic reservoir computing is an emergent technology toward beyond-Neumann computing. Although photonic reservoir computing provides superior performance in environments whose characteristics are coincident with the training datasets for the reservoir, the performance is significantly degraded if these characteristics deviate from the original knowledge used in the training phase. Here, we propose a scheme of adaptive model selection in photonic reservoir computing using reinforcement learning. In this scheme, a temporal waveform is generated by different dynamic source models that change ov…
Inferring bifurcation diagrams of two distinct chaotic systems by a single machine
School of Physics and Information Technology, Shaanxi Normal University, Xi'an 710062, China（陕西师范大学物理与信息技术学院，西安710062，中国）；College of Physics and Electronic Information Engineering, Qinghai Normal University, Xining 810008, China（青海师范大学物理与电子信息技术学院，西宁810008，中国）；School of Physics and Electronic Science, East China Normal University, Shanghai 200062, China（华东师范大学物理与电子科学学院，上海200062，中国）
Medical Imaging Classification with Cold-Atom Reservoir Computing using Auto-Encoders and Surrogate-Driven Training
Instituto de Telecomunicações, Dept. of Electrical and Computer Engineering, University of Coimbra, Portugal（葡萄牙科英布拉大学电信研究所，电气与计算机工程系）；ISR - Institute of Systems and Robotics, Dept. of Electrical and Computer Engineering, University of Coimbra, Portugal（葡萄牙科英布拉大学系统与机器人研究所，电气与计算机工程系）；CFisUC, Department of Physics, University of Coimbra, Portugal（葡萄牙科英布拉大学物理系，CFisUC）
Reservoir computing is a neural network approach for processing time-dependent signals that has seen rapid development in recent years. Physical implementations of the technique using optical reservoirs have demonstrated remarkable accuracy and processing speed at benchmark tasks. However, these approaches require an electronic output layer to maintain high performance, which limits their use in tasks such as time-series prediction, where the output is fed back into the reservoir. We present here a reservoir computing scheme that has rapid processing speed both by the reservoir and the output …
We describe a method for searching the optimal hyper-parameters in reservoir computing, which consists of a Gaussian process with Bayesian optimization. It provides an alternative to other frequently used optimization methods such as grid, random, or manual search. In addition to a set of optimal hyper-parameters, the method also provides a probability distribution of the cost function as a function of the hyper-parameters. We apply this method to two types of reservoirs: nonlinear delay nodes and echo state networks. It shows excellent performance on all considered benchmarks, either matching…
In this work, we propose a new approach towards the efficient optimization and implementation of reservoir computing hardware reducing the required domain expert knowledge and optimization effort. First, we adapt the reservoir input mask to the structure of the data via linear autoencoders. We therefore incorporate the advantages of dimensionality reduction and dimensionality expansion achieved by conventional algorithmically efficient linear algebra procedures of principal component analysis. Second, we employ evolutionary-inspired genetic algorithm techniques resulting in a highly efficient …
Potomac Research, LLC（波托马克研究有限责任公司）；University of Maryland（马里兰大学）
Recent research has established the effectiveness of machine learning for data-driven prediction of the future evolution of unknown dynamical systems, including chaotic systems. However, these approaches require large amounts of measured time series data from the process to be predicted. When only limited data is available, forecasters are forced to impose significant model structure that may or may not accurately represent the process of interest. In this work, we present a Meta-learning Approach to Reservoir Computing (MARC), a data-driven approach to automatically extract an appropriate mod…
University of New Mexico（新墨西哥大学）；Portland State University（波特兰州立大学）
Reservoir computing (RC) is a novel approach to time series prediction using recurrent neural networks. In RC, an input signal perturbs the intrinsic dynamics of a medium called a reservoir. A readout layer is then trained to reconstruct a target output from the reservoir's state. The multitude of RC architectures and evaluation metrics poses a challenge to both practitioners and theorists who study the task-solving performance and computational power of RC. In addition, in contrast to traditional computation models, the reservoir is a dynamical system in which computation and memory are insep…
Prediction of Nonlinear Oscillations in a Jumping Quarter-Car Model Using Reservoir Computing
Tokyo University of Agriculture and Technology（东京农工大学）；Amity University Haryana - Gurugram（Amity大学哈里亚纳邦-古鲁格拉姆校区）；Nagoya Institute of Technology（名古屋工业大学）；Central University of Rajasthan（中央拉贾斯坦大学）
Institute of Theoretical Physics, Technische Universität Berlin（柏林工业大学理论物理研究所）；Department of Mathematics, Humboldt-Universität zu Berlin（柏林洪堡大学数学系）；Institute of Mathematics, Technische Universität Berlin（柏林工业大学数学研究所）
The Deep Time-Delay Reservoir Computing concept utilizes unidirectionally connected systems with time-delays for supervised learning. We present how the dynamical properties of a deep Ikeda-based reservoir are related to its memory capacity (MC) and how that can be used for optimization. In particular, we analyze bifurcations of the corresponding autonomous system and compute conditional Lyapunov exponents, which measure the generalized synchronization between the input and the layer dynamics. We show how the MC is related to the systems distance to bifurcations or magnitude of the conditional…
Cooperative Institute for Research in Environmental Sciences（环境科学合作研究所）；University of Colorado Boulder（科罗拉多大学博尔德分校）；Physical Sciences Laboratory（物理科学实验室）；National Oceanic and Atmospheric Administration（美国国家海洋和大气管理局）；University of California San Diego（加利福尼亚大学圣迭戈分校）
Next generation reservoir computing based on nonlinear vector autoregression (NVAR) is applied to emulate simple dynamical system models and compared to numerical integration schemes such as Euler and the $2^\text{nd}$ order Runge-Kutta. It is shown that the NVAR emulator can be interpreted as a data-driven method used to recover the numerical integration scheme that produced the data. It is also shown that the approach can be extended to produce high-order numerical schemes directly from data. The impacts of the presence of noise and temporal sparsity in the training set is further examined t…
Classical and Quantum Physical Reservoir Computing for Onboard Artificial Intelligence Systems: A Perspective
Artificial Intelligence and Cyber Futures Institute, Charles Sturt University（查尔斯斯特大学人工智能与网络未来学院）；Cyber Futures Institute, Charles Sturt University（查尔斯斯特大学网络未来学院）
