We use standard deep neural networks to classify univariate time series generated by discrete and continuous dynamical systems based on their chaotic or non-chaotic behaviour. Our approach to circumvent the lack of precise models for some of the most challenging real-life applications is to train different neural networks on a data set from a dynamical system with a basic or low-dimensional phase space and then use these networks to classify univariate time series of a dynamical system with more intricate or high-dimensional phase space. We illustrate this generalisation approach using the logistic map, the sine-circle map, the Lorenz system, and the Kuramoto--Sivashinsky equation. We observe that a convolutional neural network without batch normalization layers outperforms state-of-the-art neural networks for time series classification and is able to generalise and classify time series as chaotic or not with high accuracy.
Inspired by the well-known permutation entropy (PE), an effective image encoding scheme for chaotic time series, Triad State Space Construction (TSSC), is proposed. The TSSC image can recognize higher-order temporal patterns and identify new forbidden regions in time series motifs beyond the Bandt-Pompe probabilities. The Convolutional Neural Network (ConvNet) is widely used in image classification. The ConvNet classifier based on TSSC images (TSSC-ConvNet) are highly accurate and very robust in the chaotic signal classification.
Karachi Institute of Economics and Technology（卡拉奇经济与技术学院）；Iqra University（伊克拉大学）
Due to the dynamic nature, chaotic time series are difficult predict. In conventional signal processing approaches signals are treated either in time or in space domain only. Spatio-temporal analysis of signal provides more advantages over conventional uni-dimensional approaches by harnessing the information from both the temporal and spatial domains. Herein, we propose an spatio-temporal extension of RBF neural networks for the prediction of chaotic time series. The proposed algorithm utilizes the concept of time-space orthogonality and separately deals with the temporal dynamics and spatial …
Artificial Intelligence Center（人工智能中心）；Czech Technical University（捷克技术大学）；Niels Bohr Institute（尼尔斯·玻尔研究所）；University of Copenhagen（哥本哈根大学）
This work builds an automated anomaly detection method for chaotic time series, and more concretely for turbulent, high-dimensional, ocean simulations. We solve this task by extending the Echo State Network by spatially aware input maps, such as convolutions, gradients, cosine transforms, et cetera, as well as a spatially aware loss function. The spatial ESN is used to create predictions which reduce the detection problem to thresholding of the prediction error. We benchmark our detection framework on different tasks of increasing difficulty to show the generality of the framework before apply…
In this work we introduce a class of dynamic models for time series taking values on the unit interval. The proposed model follows a generalized linear model approach where the random component, conditioned on the past information, follows a beta distribution, while the conditional mean specification may include covariates and also an extra additive term given by the iteration of a map that can present chaotic behavior. The resulting model is very flexible and its systematic component can accommodate short and long range dependence, periodic behavior, laminar phases, etc. We derive easily veri…
Surrogate testing techniques have been used widely to investigate the presence of dynamical nonlinearities, an essential ingredient of deterministic chaotic processes. Traditional surrogate testing subscribes to statistical hypothesis testing and investigates potential differences in discriminant statistics between the given empirical sample and its surrogate counterparts. The choice and estimation of the discriminant statistics can be challenging across short time series. Also, conclusion based on a single empirical sample is an inherent limitation. The present study proposes a recurrent neur…
One of the most useful tools for distinguishing between chaotic and stochastic time series is the so-called complexity-entropy causality plane. This diagram involves two complexity measures: the Shannon entropy and the statistical complexity. Recently, this idea has been generalized by considering the Tsallis monoparametric generalization of the Shannon entropy, yielding complexity-entropy curves. These curves have proven to enhance the discrimination among different time series related to stochastic and chaotic processes of numerical and experimental nature. Here we further explore these comp…
Central Institute of Mental Health（中央心理健康研究所）；Heidelberg University（海德堡大学）
Recurrent neural networks (RNNs) are wide-spread machine learning tools for modeling sequential and time series data. They are notoriously hard to train because their loss gradients backpropagated in time tend to saturate or diverge during training. This is known as the exploding and vanishing gradient problem. Previous solutions to this issue either built on rather complicated, purpose-engineered architectures with gated memory buffers, or - more recently - imposed constraints that ensure convergence to a fixed point or restrict (the eigenspectrum of) the recurrence matrix. Such constraints, …
Université de Toulouse（图卢兹大学）；CNRS（法国国家科学研究中心）；UPS（图卢兹第三大学）
We explore the influence of precision of the data and the algorithm for the simulation of chaotic dynamics by neural networks techniques. For this purpose, we simulate the Lorenz system with different precisions using three different neural network techniques adapted to time series, namely reservoir computing (using ESN), LSTM and TCN, for both short and long time predictions, and assess their efficiency and accuracy. Our results show that the ESN network is better at predicting accurately the dynamics of the system, and that in all cases the precision of the algorithm is more important than t…
Adaptive Nonlinear Vector Autoregression: Robust Forecasting for Noisy Chaotic Time Series
Department of Mathematics, Pusan National University, Republic of Korea（釜山国立大学数学系）；Pusan National University, Republic of Korea（釜山国立大学）；Industrial Mathematics Center, Pusan National University, Republic of Korea（釜山国立大学工业数学中心）；Humanoid Olfactory Display Center, Pusan National University, Republic of Korea（釜山国立大学人形嗅觉显示中心）；Department of Mathematics, Yonsei University, Republic of Korea（成均馆大学数学系）；Innovation Center for MathScience Research&Education, Pusan National University, Republic of Korea（釜山国立大学数学科学研究与教育创新中心）；Marine Natural Disaster Research Department, Korea Institute of Ocean Science and Technology, Republic of Korea（韩国海洋科学与技术研究院海洋自然灾害研究部）；Institute for Future Earth, Pusan National University, Republic of Korea（未来地球研究所）
Two recent papers on prediction of chaotic systems, one on multi-view embedding1 , and the second on prediction in projection2 provide empirical evidence to support particular prediction methods for chaotic systems. Multi-view embedding1 is a method of using several multivariate time series to come up with an improved embedding based predictor of a chaotic time series. Prediction in projection2 discusses how much smaller embeddings can provide useful prediction even though they may not be able to resolve the dynamics of the system. Both papers invoke a nearest neighbor3, or Lorenz method of An…
Potomac Research, LLC（波托马克研究有限责任公司）；University of Maryland（马里兰大学）
Recent research has established the effectiveness of machine learning for data-driven prediction of the future evolution of unknown dynamical systems, including chaotic systems. However, these approaches require large amounts of measured time series data from the process to be predicted. When only limited data is available, forecasters are forced to impose significant model structure that may or may not accurately represent the process of interest. In this work, we present a Meta-learning Approach to Reservoir Computing (MARC), a data-driven approach to automatically extract an appropriate mod…
Causal analysis based on non-uniform embedding schemes is an important way to detect the underlying interactions between dynamic systems. However, there are still some obstacles to estimate high-dimensional conditional mutual information and form optimal mixed embedding vector in traditional non-uniform embedding schemes. In this study, we present a new non-uniform embedding method framed in information theory to detect causality for multivariate time series, named LM-PMIME, which integrates the low-dimensional approximation of conditional mutual information and the mixed search strategy for t…
Neural Network Entropy (NNetEn): Entropy-Based EEG Signal and Chaotic Time Series Classification, Python Package for NNetEn Calculation
Institute of Physics and Technology, Petrozavodsk State University（彼得罗扎沃茨克国立大学物理与技术学院）；Kuwait College of Science and Technology（科威特科学技术学院）；Vels Institute of Sciences, Technology, and Advanced Studies（维尔斯科学技术与高等研究院）；Damghan University（达姆甘大学）
This study suggests a new prediction model for chaotic time series inspired by the brain emotional learning of mammals. We describe the structure and function of this model, which is referred to as BELPM (Brain Emotional Learning-Based Prediction Model). Structurally, the model mimics the connection between the regions of the limbic system, and functionally it uses weighted k nearest neighbors to imitate the roles of those regions. The learning algorithm of BELPM is defined using steepest descent (SD) and the least square estimator (LSE). Two benchmark chaotic time series, Lorenz and Henon, ha…
In this work, a non intrusive load disaggregation scheme is proposed. By using a kernel based nonlinear regression strategy, the switching dynamic of an electric network, simulated as a set of RLC circuits with chaotic switching, is approximated using a time series of the total power consumption. The results suggest that the employed methodology can be useful in the design of efficient load disaggregation schemes.
The striking fractal geometry of strange attractors underscores the generative nature of chaos: like probability distributions, chaotic systems can be repeatedly measured to produce arbitrarily-detailed information about the underlying attractor. Chaotic systems thus pose a unique challenge to modern statistical learning techniques, while retaining quantifiable mathematical properties that make them controllable and interpretable as benchmarks. Here, we present a growing database currently comprising 131 known chaotic dynamical systems spanning fields such as astrophysics, climatology, and bio…
Dynamic channel selection is among the most important wireless communication elements in dynamically changing electromagnetic environments wherein a user can experience improved communication quality by choosing a better channel. Multi-armed bandit (MAB) algorithms are a promising approach by which the difficult tradeoff between exploration to search for better a channel and exploitation to experience enhanced communication quality is resolved. Ultrafast solution of MAB problems has been demonstrated by utilizing chaotically oscillating time series generated by semiconductor lasers. In this st…
