This paper demonstrates the application of Bayesian Artificial Neural Networks to Ordinary Differential Equation (ODE) inverse problems. We consider the case of estimating an unknown chaotic dynamical system transition model from state observation data. Inverse problems for chaotic systems are numerically challenging as small perturbations in model parameters can cause very large changes in estimated forward trajectories. Bayesian Artificial Neural Networks can be used to simultaneously fit a model and estimate model parameter uncertainty. Knowledge of model parameter uncertainty can then be incorporated into the probabilistic estimates of the inferred system's forward time evolution. The method is demonstrated numerically by analysing the chaotic Sprott B system. Observations of the system are used to estimate a posterior predictive distribution over the weights of a parametric polynomial kernel Artificial Neural Network. It is shown that the proposed method is able to perform accurate time predictions. Further, the proposed method is able to correctly account for model uncertainties and provide useful prediction uncertainty bounds.
The Dynamic-Probabilistic Consistency Gap in Chaotic Surrogate Modeling
Interdisciplinary Center for Scientific Computing, Heidelberg University, Germany（海德堡大学交叉科学计算中心）；Faculty of Mathematics and Computer Science, Heidelberg University, Germany（海德堡大学数学与计算机科学学院）；Dept. of Theoretical Neuroscience, Central Institute of Mental Health (CIMH), Mannheim, Germany（曼海姆中央心理健康研究所理论神经科学部门）；Faculty of Physics and Astronomy, Heidelberg University, Germany（海德堡大学物理与天文学学院）；Hector Institute for AI in Psychiatry and Dept. of Psychiatry and Psychotherapy, CIMH, Mannheim, Germany（曼海姆中央心理健康研究所精神病AI研究所及精神病学与心理学系）；Hertie Institute for AI in Brain Health, University of Tübingen, Germany（图宾根大学脑健康AI研究所）
Being in the era of big data, modeling and prediction of count data have become significantly important in many fields including health, finance, social, etc. Although linear Poisson regression has been widely used to model count and rate data, it might not be always suitable as it cannot capture some inherent variability within complex data. In this study, we introduce a probabilistically driven nonlinear Poisson regression model with Bayesian artificial neural networks (ANN) to model count or rate data. This new nonlinear Poisson regression model developed with Bayesian ANN provides higher p…
National Institute of Advanced Studies（国家高等研究院）；PES University（PES大学）
Inspired by chaotic firing of neurons in the brain, we propose ChaosNet -- a novel chaos based artificial neural network architecture for classification tasks. ChaosNet is built using layers of neurons, each of which is a 1D chaotic map known as the Generalized Luroth Series (GLS) which has been shown in earlier works to possess very useful properties for compression, cryptography and for computing XOR and other logical operations. In this work, we design a novel learning algorithm on ChaosNet that exploits the topological transitivity property of the chaotic GLS neurons. The proposed learning…
The use of artificial neural networks as models of chaotic dynamics has been rapidly expanding. Still, a theoretical understanding of how neural networks learn chaos is lacking. Here, we employ a geometric perspective to show that neural networks can efficiently model chaotic dynamics by becoming structurally chaotic themselves. We first confirm neural network's efficiency in emulating chaos by showing that a parsimonious neural network trained only on few data points can reconstruct strange attractors, extrapolate outside training data boundaries, and accurately predict local divergence rates…
Controlling Chaos could be a big factor in getting great stable amounts of energy out of small amounts of not necessarily stable resources. By definition, Chaos is getting huge changes in the system's output due to unpredictable small changes in initial conditions, and that means we could take advantage of this fact and select the proper control system to manipulate system's initial conditions and inputs in general and get a desirable output out of otherwise a Chaotic system. That was accomplished by first building some known chaotic circuit (Chua circuit) and the NI's MultiSim was used to sim…
On the Predictive Skill of Artificial Intelligence-based Weather Models for Extreme Events using Uncertainty Quantification
Applied Machine Learning Group（应用机器学习组）；Fraunhofer Heinrich-Hertz Institute（弗劳恩霍夫海因里希-赫兹研究所）；Signal Theory and Communications Dept.（信号理论与通信系）；Universidad Carlos III de Madrid (UC3M)（马德里卡洛斯三世大学）
Probabilistic Forecasting with Generative Networks via Scoring Rule Minimization
University of Oxford（牛津大学）；Southern University of Science and Technology（南方科技大学）；University of Warwick（华威大学）；European Centre for Medium-Range Weather Forecasts（欧洲中期天气预报中心）
The University of Tokyo（东京大学）；Kobe University（神户大学）；Saitama University（埼玉大学）
Generative adversarial network (GAN) is gaining increased importance in artificially constructing natural images and related functionalities wherein two networks called generator and discriminator are evolving through adversarial mechanisms. Using deep convolutional neural networks and related techniques, high-resolution, highly realistic scenes, human faces, among others have been generated. While GAN in general needs a large amount of genuine training data sets, it is noteworthy that vast amounts of pseudorandom numbers are required. Here we utilize chaotic time series generated experimental…
Universidade Federal Rural de Pernambuco（伯南布哥联邦农村大学）；Universidade Federal de Pernambuco（伯南布哥联邦大学）
Probabilistic Quantum Memory (PQM) is a data structure that computes the distance from a binary input to all binary patterns stored in superposition on the memory. This data structure allows the development of heuristics to speed up artificial neural networks architecture selection. In this work, we propose an improved parametric version of the PQM to perform pattern classification, and we also present a PQM quantum circuit suitable for Noisy Intermediate Scale Quantum (NISQ) computers. We present a classical evaluation of a parametric PQM network classifier on public benchmark datasets. We al…
Institute of Electrical Engineering “Nikola Tesla”（“尼古拉·特斯拉”电气工程研究所）；EE Institute “Nikola Tesla”（“尼古拉·特斯拉”电气工程研究所）
In this paper, we introduce elements of probabilistic model that is suitable for modeling of learning algorithms in biologically plausible artificial neural networks framework. Model is based on two of the main concepts in quantum physics - a density matrix and the Born rule. As an example, we will show that proposed probabilistic interpretation is suitable for modeling of on-line learning algorithms for PSA, which are preferably realized by a parallel hardware based on very simple computational units. Proposed concept (model) can be used in the context of improving algorithm convergence speed…
Object selection refers to the mechanism of extracting objects of interest while ignoring other objects and background in a given visual scene. It is a fundamental issue for many computer vision and image analysis techniques and it is still a challenging task to artificial visual systems. Chaotic phase synchronization takes place in cases involving almost identical dynamical systems and it means that the phase difference between the systems is kept bounded over the time, while their amplitudes remain chaotic and may be uncorrelated. Instead of complete synchronization, phase synchronization is…
A Compression Based Classification Framework Using Symbolic Dynamics of Chaotic Maps
Department of Computer Science and Information Systems (CSIS)（计算机科学与信息系统系）；Adjunct Faculty, Consciousness Studies Programme, National Institute of Advanced Studies, IISc Campus, Bengaluru, 560012, Karnataka, India（意识研究项目兼职教员，国家高级研究院，IISc校区，班加罗尔，560012，卡纳塔克邦，印度）
Spiking neural networks (SNNs) are distributed trainable systems whose computing elements, or neurons, are characterized by internal analog dynamics and by digital and sparse synaptic communications. The sparsity of the synaptic spiking inputs and the corresponding event-driven nature of neural processing can be leveraged by energy-efficient hardware implementations, which can offer significant energy reductions as compared to conventional artificial neural networks (ANNs). The design of training algorithms lags behind the hardware implementations. Most existing training algorithms for SNNs ha…
Rolls-Royce plc（罗尔斯·罗伊斯公司）；University of Cambridge（剑桥大学）
An artificial intelligence tool is exploited to discover and characterize a new molybdenum-base alloy that is the most likely to simultaneously satisfy targets of cost, phase stability, precipitate content, yield stress, and hardness. Experimental testing demonstrates that the proposed alloy fulfils the computational predictions, and furthermore the physical properties exceed those of other commercially available Mo-base alloys for forging-die applications.
It is well known that artificial neural networks (ANNs) can learn deterministic automata. Learning nondeterministic automata is another matter. This is important because much of the world is nondeterministic, taking the form of unpredictable or probabilistic events that must be acted upon. If ANNs are to engage such phenomena, then they must be able to learn how to deal with nondeterminism. In this project the game of Pong poses a nondeterministic environment. The learner is given an incomplete view of the game state and underlying deterministic physics, resulting in a nondeterministic game. T…
The goal of this paper is to determine the laws of observed trajectories assuming that there is a mechanical system in the background and using these laws to continue the observed motion in a plausible way. The laws are represented by neural networks with a limited number of parameters. The training of the networks follows the Extreme Learning Machine idea. We determine laws for different levels of embedding, thus we can represent not only the equation of motion but also the symmetries of different kinds. In the recursive numerical evolution of the system, we require the fulfillment of all the…
Tensor-Networks-based Learning of Probabilistic Cellular Automata Dynamics
Singapore University of Technology and Design（新加坡科技设计大学）；Okinawa Institute of Science and Technology Graduate University（冲绳科学技术大学院大学）；Hunan Normal University（湖南师范大学）；National University of Singapore（新加坡国立大学）；CNRS-UNS-NUS-NTU International Joint Research Unit（法国国家科学研究中心-新南威尔士大学-新加坡国立大学-南洋理工大学国际联合研究单位）
University of Edinburgh（爱丁堡大学）；University of Cambridge（剑桥大学）；Instituto de Telecomunicações（电信研究所）；Leiden University（莱顿大学）
Since its formulation by Sir Isaac Newton, the problem of solving the equations of motion for three bodies under their own gravitational force has remained practically unsolved. Currently, the solution for a given initialization can only be found by performing laborious iterative calculations that have unpredictable and potentially infinite computational cost, due to the system's chaotic nature. We show that an ensemble of solutions obtained using an arbitrarily precise numerical integrator can be used to train a deep artificial neural network (ANN) that, over a bounded time interval, provides…
Chemical Mass Casualty Incidents (MCI) place a heavy burden on hospital staff and resources. Machine Learning (ML) tools can provide efficient decision support to caregivers. However, ML models require large volumes of data for the most accurate results, which is typically not feasible in the chaotic nature of a chemical MCI. This study examines the application of four statistical dimension reduction techniques: Random Selection, Covariance/Variance, Pearson's Linear Correlation, and Principle Component Analysis to reduce a dataset of 311 hazardous chemicals and 79 related signs and symptoms (…
