We test four fast mean field type algorithms on Hopfield networks as an inverse Ising problem. The equilibrium behavior of Hopfield networks is simulated through Glauber dynamics. In the low temperature regime, the simulated annealing technique is adopted. Although performances of these network reconstruction algorithms on the simulated network of spiking neurons are extensively studied recently, the analysis of Hopfield networks is lacking so far. For the Hopfield network, we found that, in the retrieval phase favored when the network wants to memory one of stored patterns, all the reconstruction algorithms fail to extract interactions within a desired accuracy, and the same failure occurs in the spin glass phase where spurious minima show up, while in the paramagnetic phase, albeit unfavored during the retrieval dynamics, the algorithms work well to reconstruct the network itself. This implies that, as a inverse problem, the paramagnetic phase is conversely useful for reconstructing the network while the retrieval phase loses all the information about interactions in the network except for the case where only one pattern is stored. The performances of algorithms are studied with respect to the system size, memory load and temperature, sample-to-sample fluctuations are also considered.
Hopfield model is one of the few neural networks for which analytical results can be obtained. However, most of them are derived under the assumption of random uncorrelated patterns, while in real life applications data to be stored show non-trivial correlations. In the present paper we study how the retrieval capability of the Hopfield network at null temperature is affected by spatial correlations in the data we feed to it. In particular, we use as patterns to be stored the configurations of a linear Ising model at inverse temperature $β$. Exploiting the signal to noise technique we obtain a…
Spin glasses, generally defined as disordered systems with randomized competing interactions, are a widely investigated complex system. Theoretical models describing spin glasses are broadly used in other complex systems, such as those describing brain function, error-correcting codes, or stock-market dynamics. This wide interest in spin glasses provides strong motivation to generate an artificial spin glass within the framework of artificial spin ice systems. Here, we present the experimental realization of an artificial spin glass consisting of dipolar coupled single-domain Ising-type nanoma…
The coherent Ising machine is expected to find a near-optimal solution in various combinatorial optimization problems, which has been experimentally confirmed with optical parametric oscillators (OPOs) and a field programmable gate array (FPGA) circuit. The similar mathematical models were proposed three decades ago by J. J. Hopfield, et al. in the context of classical neural networks. In this article, we compare the computational performance of both models.
We explore the question as to whether quantum effects can yield a speedup of the non-equilibrium evolution of spin systems towards a classical thermal state. In our approach we exploit the fact that the thermal state of a spin system can be mapped onto a node-free quantum state whose coefficients are given by thermal weights. This perspective permits the construction of a dissipative -- yet quantum -- dynamics which encodes in its stationary state the thermal state of the original problem. We show for the case of an all-to-all connected Ising spin model that an appropriate transformation of th…
Statistical mechanics of vector Hopfield network near and above saturation
Chalmers University of Technology and University of Gothenburg（查尔姆斯理工大学和哥德堡大学）；Sapienza Università di Roma（罗马第一大学）；National Research Council of Italy, CNR - NANOTEC, Rome Unit（意大利国家研究委员会纳米技术研究所罗马分部）
We study a class of Hopfield models where the memories are represented by a mixture of Gaussian and binary variables and the neurons are Ising spins. We study the properties of this family of models as the relative weight of the two kinds of variables in the patterns varies. We quantitatively determine how the retrieval phase squeezes towards zero as the memory patterns contain a larger fraction of mismatched variables. As the memory is purely Gaussian retrieval is lost for any positive storage capacity. It is shown that this comes about because of the spherical symmetry of the free energy in …
In this work we explain how to properly use mean-field methods to solve the inverse Ising problem when the phase space is clustered, that is many states are present. The clustering of the phase space can occur for many reasons, e.g. when a system undergoes a phase transition. Mean-field methods for the inverse Ising problem are typically used without taking into account the eventual clustered structure of the input configurations and may led to very bad inference (for instance in the low temperature phase of the Curie-Weiss model). In the present work we explain how to modify mean-field approa…
We propose the Wishart planted ensemble, a class of zero-field Ising models with tunable algorithmic hardness and specifiable (or planted) ground state. The problem class arises from a simple procedure for generating a family of random integer programming problems with specific statistical symmetry properties, but turns out to have intimate connections to a sign-inverted variant of the Hopfield model. The Hamiltonian contains only 2-spin interactions, with the coupler matrix following a type of Wishart distribution. The class exhibits a classical first-order phase transition in temperature. Fo…
This article is intended for physical scientists who wish to gain deeper insights into machine learning algorithms which we present via the domain they know best, physics. We begin with a review of two energy-based machine learning algorithms, Hopfield networks and Boltzmann machines, and their connection to the Ising model. This serves as a foundation to understand the phenomenon of learning more generally. Equipped with this intuition we then delve into additional, more "practical," machine learning architectures including feedforward neural networks, convolutional neural networks, and autoe…
We develop efficient ways to consider and correct for the effects of hidden units for the paradigmatic case of the inverse kinetic Ising model with fully asymmetric couplings. We identify two sources of error in reconstructing the connectivity among the observed units while ignoring part of the network. One leads to a systematic bias in the inferred parameters, whereas the other involves correlations between the visible and hidden populations and has a magnitude that depends on the coupling strength. We estimate these two terms using a mean field approach and derive self-consistent equations f…
Phases in a class of associative memories via hidden neurons
CyberAgent AI Lab（CyberAgent AI实验室）；RIKEN iTHEMS（RIKEN跨学科理论科学研究所）；Graduate School of Artificial Intelligence and Science, Rikkyo University（立教大学人工智能与科学研究科）
Observation of associative-memory retrieval and spin-glass phases on a photonic quantum simulator
Sapienza Università di Roma（罗马大学）；Università del Salento（萨伦托大学）；Institute of Nanotechnology of the National Research Council of Italy, CNR-NANOTEC（意大利国家研究委员会纳米技术研究所）；Università degli Studi di Salerno（萨莱诺大学）；INFN, Sezione di Napoli, Gruppo Collegato di Salerno（意大利国家核物理研究所那不勒斯分部萨莱诺协作组）；Department of Computing Sciences, Bocconi University（博科尼大学计算科学系）；Istituto di Fotonica e Nanotecnologie, Consiglio Nazionale delle Ricerche (IFN-CNR)（意大利国家研究委员会光子和纳米技术研究所）
