Brains process information through the collective dynamics of large neural networks. Collective chaos was suggested to underlie the complex ongoing dynamics observed in cerebral cortical circuits and determine the impact and processing of incoming information streams. In dissipative systems, chaotic dynamics takes place on a subset of phase space of reduced dimensionality and is organized by a complex tangle of stable, neutral and unstable manifolds. Key topological invariants of this phase space structure such as attractor dimension, and Kolmogorov-Sinai entropy so far remained elusive.
Here we calculate the complete Lyapunov spectrum of recurrent neural networks. We show that chaos in these networks is extensive with a size-invariant Lyapunov spectrum and characterized by attractor dimensions much smaller than the number of phase space dimensions. We find that near the onset of chaos, for very intense chaos, and discrete-time dynamics, random matrix theory provides analytical approximations to the full Lyapunov spectrum. We show that a generalized time-reversal symmetry of the network dynamics induces a point-symmetry of the Lyapunov spectrum reminiscent of the symplectic structure of chaotic Hamiltonian systems. Fluctuating input reduces both the entropy rate and the attractor dimension. For trained recurrent networks, we find that Lyapunov spectrum analysis provides a quantification of error propagation and stability achieved. Our methods apply to systems of arbitrary connectivity, and we describe a comprehensive set of controls for the accuracy and convergence of Lyapunov exponents.
Our results open a novel avenue for characterizing the complex dynamics of recurrent neural networks and the geometry of the corresponding chaotic attractors. They also highlight the potential of Lyapunov spectrum analysis as a diagnostic for machine learning applications of recurrent networks.
In this article, on the example of the known low-order dynamical models, namely Lorenz, Rossler and Vallis systems, the difficulties of reliable numerical analysis of chaotic dynamical systems are discussed. For the Lorenz system, the problems of existence of hidden chaotic attractors and hidden transient chaotic sets and their numerical investigation are considered. The problems of the numerical characterization of a chaotic attractor by calculating finite-time time Lyapunov exponents and finite-time Lyapunov dimension along one trajectory are demonstrated using the example of computing unsta…
Dynamical billiards are paradigmatic examples of chaotic Hamiltonian dynamical systems with widespread applications in physics. We study how well their Lyapunov exponent, characterizing the chaotic dynamics, and its dependence on external parameters can be estimated from phase space volume arguments, with emphasis on billiards with mixed regular and chaotic phase spaces. We show that in the very diverse billiards considered here the leading contribution to the Lyapunov exponent is inversely proportional to the chaotic phase space volume, and subsequently discuss the generality of this relation…
Despite the prominent importance of the Lyapunov exponents for characterizing chaos, it still remains a challenge to measure them for large experimental systems, mainly because of the lack of recurrences in time series analysis. Here we develop a method to overcome this difficulty, valid for highly symmetric systems such as systems with global coupling for which the dimensionality of recurrence analysis can be reduced drastically. We test our method numerically with two globally coupled systems, namely, logistic maps and limit-cycle oscillators with global coupling. The evaluated exponent valu…
The sensitive dependence of chaos on parameters is a topic of great interest in the study of integrability and stability of dynamical systems. Previous work has proposed ways to identify the sensitive dependence on parameters by topological criteria or large numerical simulations. In this paper, we show that when the Lyapunov exponents of the system vary with a change in the parameters, the system diverges exponentially in the orbits associated with the considered parameters. We use this result to explore the sensitive dependence on parameters in an uncertainty interval and conclude that the c…
We study chaotic dynamics in a system of four differential equations describing the dynamics of five identical globally coupled phase oscillators with biharmonic coupling. We show that this system exhibits strange spiral attractors (Shilnikov attractors) with two zero (indistinguishable from zero in numerics) Lyapunov exponents in a wide region of the parameter space. We explain this phenomenon by means of bifurcation analysis of the three-dimensional Poincaré map for the system under consideration. We show that the chaotic dynamics develop here near a codimension three bifurcation, when a per…
On the relation between reliable computation time, float-point precision and the Lyapunov exponent in chaotic systems
Center for Monsoon System Research, Institute of Atmospheric Physics, Chinese Academy of Sciences（中国科学院大气物理研究所季风系统研究中心）；State Key Laboratory of Numerical Modeling for Atmospheric Sciences and Geophysical Fluid Dynamics, Institute of Atmospheric Physics, Chinese Academy of Sciences（中国科学院大气物理研究所大气数值模拟和地球流体力学国家重点实验室）；College of Global Change and Earth System Science, Beijing Normal University（北京师范大学全球变化与地球系统科学学院）
The Kuramoto-Sivashinsky equation is a prototypical chaotic nonlinear partial differential equation (PDE) in which the size of the spatial domain plays the role of a bifurcation parameter. We investigate the changing dynamics of the Kuramoto-Sivashinsky PDE by calculating the Lyapunov spectra over a large range of domain sizes. Our comprehensive computation and analysis of the Lyapunov exponents and the associated Kaplan-Yorke dimension provides new insights into the chaotic dynamics of the Kuramoto-Sivashinsky PDE, and the transition to its 1D turbulence.
We characterize the dynamical states of a piezoelectric microelectromechanical system (MEMS) using several numerical quantifers including the maximal Lyapunov exponent, the Poincare Surface of Section and a chaos detection method called the Smaller Alignment Index (SALI). The analysis makes use of the MEMS Hamiltonian. We start our study by considering the case of a conservative piezoelectric MEMS model and describe the behavior of some representative phase space orbits of the system. We show that the dynamics of the piezoelectric MEMS becomes considerably more complex as the natural frequency…
