Kinetic Ising models are powerful tools for studying the non-equilibrium dynamics of complex systems. As their behavior is not tractable for large networks, many mean-field methods have been proposed for their analysis, each based on unique assumptions about the system's temporal evolution. This disparity of approaches makes it challenging to systematically advance mean-field methods beyond previous contributions. Here, we propose a unifying framework for mean-field theories of asymmetric kinetic Ising systems from an information geometry perspective. The framework is built on Plefka expansions of a system around a simplified model obtained by an orthogonal projection to a sub-manifold of tractable probability distributions. This view not only unifies previous methods but also allows us to develop novel methods that, in contrast with traditional approaches, preserve the system's correlations. We show that these new methods can outperform previous ones in predicting and assessing network properties near maximally fluctuating regimes.
We develop an elementary mean field approach for fully asymmetric kinetic Ising models, which can be applied to a single instance of the problem. In the case of the asymmetric SK model this method gives the exact values of the local magnetizations and the exact relation between equal-time and time-delayed correlations. It can also be used to solve efficiently the inverse problem, i.e. determine the couplings and local fields from a set of patterns, also in cases where the fields and couplings are time-dependent. This approach generalizes some recent attempts to solve this dynamical inference p…
We study analytically the performance of a recently proposed algorithm for learning the couplings of a random asymmetric kinetic Ising model from finite length trajectories of the spin dynamics. Our analysis shows the importance of the nontrivial equal time correlations between spins induced by the dynamics for the speed of learning. These correlations become more important as the spin's stochasticity is decreased. We also analyse the deviation of the estimation error from asymptotic optimality.
The dynamics of an asymmetric kinetic Ising model is studied. Two schemes for improving the existing mean-field description are proposed. In the first scheme, we derive the formulas for instantaneous magnetization, equal-time correlation, and time-delayed correlation, considering the correlation between different local fields. To derive the time-delayed correlation, we emphasize that the small correlation assumption adopted in previous work [M. Mézard and J. Sakellariou, J. Stat. Mech., L07001 (2011)] is in fact not required. To confirm the inference efficiency of our method, we perform extens…
The one-step replica symmetry breaking (RSB) is used to study a two-sublattice fermionic infinite-range Ising spin glass (SG) model in a transverse field $Γ$. The problem is formulated in a Grassmann path integral formalism within the static approximation. In this model, a parallel magnetic field $H$ breaks the symmetry of the sublattices. It destroys the antiferromagnetic (AF) order, but it can favor the nonergodic mixed phase (SG+AF) characterizing an asymmetric RSB region. In this region, intra-sublattice disordered interactions $V$ increase the difference between the RSB solutions of each …
We compare dynamic mean-field and dynamic cavity as methods to describe the stationary states of dilute kinetic Ising models. We compute dynamic mean-field theory by expanding in interaction strength to third order, and compare to the exact dynamic mean-field theory for fully asymmetric networks. We show that in diluted networks the dynamic cavity method generally predicts magnetizations of individual spins better than both first order ("naive") and second order ("TAP") dynamic mean field theory.
We perform a numerical study of the three-dimensional Random Field Ising Model at T=0. We compare work distributions along metastable trajectories obtained with the single-spin flip dynamics with the distribution of the internal energy change along equilibrium trajectories. The goal is to investigate the possibility of extending the Crooks fluctuation theorem to zero temperature when, instead of the standard ensemble statistics, one considers the ensemble generated by the quenched disorder. We show that a simple extension of Crooks fails close to the disordered induced equilibrium phase transi…
This contribution reviews the parallel dynamics of Q-Ising neural networks for various architectures: extremely diluted asymmetric, layered feedforward, extremely diluted symmetric, and fully connected. Using a probabilistic signal-to-noise ratio analysis, taking into account all feedback correlations, which are strongly dependent upon these architectures the evolution of the distribution of the local field is found. This leads to a recursive scheme determining the complete time evolution of the order parameters of the network. Arbitrary Q and mainly zero temperature are considered. For the as…
We study the nonconserved phase ordering dynamics of the d = 2, 3 random field Ising model, quenched to below the critical temperature. Motivated by the puzzling results of previous work in two and three di- mensions, reporting a crossover from power-law to logarithmic growth, together with superuniversal behavior of the correlation function, we have undertaken a careful investigation of both the domain growth law and the autocorrelation function. Our main results are as follows: We confirm the crossover to asymptotic logarithmic behavior in the growth law, but, at variance with previous findi…
We study three kinetic models with constraint, namely the Symmetrically Constrained Ising Chain, the Asymmetrically Constrained Ising Chain, and the Backgammon Model. All these models show glassy behavior and coarsening. We apply to them the Stillinger and Weber decomposition, and find that they share the same configurational entropy, despite of their different nonequilibrium dynamics. We conclude therefore that the Stillinger and Weber decomposition is not relevant for this type of models.
In view of the recently seen dramatic effect of quenched random bonds on tricritical systems, we have conducted a renormalization-group study on the effect of quenched random fields on the tricritical phase diagram of the spin-1 Ising model in $d=3$. We find that random fields convert first-order phase transitions into second-order, in fact more effectively than random bonds. The coexistence region is extremely flat, attesting to an unusually small tricritical exponent $β_u$; moreover, an extreme asymmetry of the phase diagram is very striking. To accomodate this asymmetry, the second-order bo…
We introduce the transverse Ising model as a prototype for discussing quantum phase transition. Next we introduce Suzuki-Trotter formalism to show the correspondence between $d$-dimensional quantum system with a $(d+1)$-dimensional classical system. We then discuss transverse Ising spin glass models, namely S-K model, E-A model, and the $\pm J$ model with Ising spin in transverse field. We briefly discuss the mean field, exact diagonalization, and quantum Monte Carlo results for their phase diagrams. Next we discuss the question of replica symmetry restoration in quantum spin glasses (due to t…
The parallel dynamics of extremely diluted symmetric Q-Ising neural networks is studied for arbitrary Q using a probabilistic approach. In spite of the extremely diluted architecture the feedback correlations arising from the symmetry prevent a closed-form solution in contrast with the extremely diluted asymmetric model. A recursive scheme is found determining the complete time evolution of the order parameters taking into account all feedback. It is based upon the evolution of the distribution of the local field, as in the fully connected model. As an illustrative example an explicit analysis…
Drift-Diffusion Matching: Embedding dynamics in latent manifolds of asymmetric neural networks
Mathematical Institute, University of Oxford（牛津大学数学研究所）；Centre for Eudaimonia and Human Flourishing, University of Oxford（牛津大学幸福与人类繁荣中心）；Complexity Science Hub, Vienna（维也纳复杂科学中心）
Randomly coupled Ising spins constitute the classical model of collective phenomena in disordered systems, with applications covering ferromagnetism, combinatorial optimization, protein folding, stock market dynamics, and social dynamics. The phase diagram of these systems is obtained in the thermodynamic limit by averaging over the quenched randomness of the couplings. However, many applications require the statistics of activity for a single realization of the possibly asymmetric couplings in finite-sized networks. Examples include reconstruction of couplings from the observed dynamics, lear…
We extend previous mean-field approaches for non-equilibrium neural network models to estimate correlations in the system. This offers a powerful tool for approximating the system dynamics as well as a fast method to infer network parameters from observations. We develop our method in an asymmetric kinetic Ising model and test its performance on 1) synthetic data generated by an asymmetric version of the Sherrington Kirkpatric model and 2) recordings of in vitro neuron spiking activity from the mouse somatosensory cortex. We find that our mean-field method outperforms previous ones in estimati…
We develop efficient ways to consider and correct for the effects of hidden units for the paradigmatic case of the inverse kinetic Ising model with fully asymmetric couplings. We identify two sources of error in reconstructing the connectivity among the observed units while ignoring part of the network. One leads to a systematic bias in the inferred parameters, whereas the other involves correlations between the visible and hidden populations and has a magnitude that depends on the coupling strength. We estimate these two terms using a mean field approach and derive self-consistent equations f…
Network structures are reconstructed from dynamical data by respectively naive mean field (nMF) and Thouless-Anderson-Palmer (TAP) approximations. For TAP approximation, we use two methods to reconstruct the network: a) iteration method; b) casting the inference formula to a set of cubic equations and solving it directly. We investigate inference of the asymmetric Sherrington- Kirkpatrick (S-K) model using asynchronous update. The solutions of the sets cubic equation depend of temperature T in the S-K model, and a critical temperature Tc is found around 2.1. For T < Tc, the solutions of the cu…
Transfer-matrix methods are used to study the probability distributions of spin-spin correlation functions $G$ in the two-dimensional random-field Ising model, on long strips of width $L = 3 - 15$ sites, for binary field distributions at generic distance $R$, temperature $T$ and field intensity $h_0$. For moderately high $T$, and $h_0$ of the order of magnitude used in most experiments, the distributions are singly-peaked, though rather asymmetric. For low temperatures the single-peaked shape deteriorates, crossing over towards a double-$δ$ ground-state structure. A connection is obtained betw…
We describe and analyze some novel approaches for studying the dynamics of Ising spin glass models. We first briefly consider the variational approach based on minimizing the Kullback-Leibler divergence between independent trajectories and the real ones and note that this approach only coincides with the mean field equations from the saddle point approximation to the generating functional when the dynamics is defined through a logistic link function, which is the case for the kinetic Ising model with parallel update. We then spend the rest of the paper developing two ways of going beyond the s…
