We here unify the field theoretical approach to neuronal networks with large deviations theory. For a prototypical random recurrent network model with continuous-valued units, we show that the effective action is identical to the rate function and derive the latter using field theory. This rate function takes the form of a Kullback-Leibler divergence which enables data-driven inference of model parameters and calculation of fluctuations beyond mean-field theory. Lastly, we expose a regime with fluctuation-induced transitions between mean-field solutions.
Neuronal networks provide living organisms with the ability to process information. They are also characterized by abundant recurrent connections, which give rise to strong feedback that dictates their dynamics and endows them with fading (short-term) memory. The role of recurrence in long-term memory, on the other hand, is still unclear. Here we use the neuronal network of the roundworm C. elegans to show that recurrent architectures in living organisms can exhibit long-term memory without relying on specific hard-wired modules. A genetic algorithm reveals that the experimentally observed dyn…
Cross-correlations in the activity in neural networks are commonly used to characterize their dynamical states and their anatomical and functional organizations. Yet, how these latter network features affect the spatiotemporal structure of the correlations in recurrent networks is not fully understood. Here, we develop a general theory for the emergence of correlated neuronal activity from the dynamics in strongly recurrent networks consisting of several populations of binary neurons. We apply this theory to the case in which the connectivity depends on the anatomical or functional distance be…
Through evolution, animals have acquired central nervous systems (CNSs), which are extremely efficient information processing devices that improve an animal's adaptability to various environments. It has been proposed that the process of information maximization (infomax), which maximizes the information transmission from the input to the output of a feedforward network, may provide an explanation of the stimulus selectivity of neurons in CNSs. However, CNSs contain not only feedforward but also recurrent synaptic connections, and little is known about information retention over time in such r…
Understanding capabilities and limitations of different network architectures is of fundamental importance to machine learning. Bayesian inference on Gaussian processes has proven to be a viable approach for studying recurrent and deep networks in the limit of infinite layer width, $n\to\infty$. Here we present a unified and systematic derivation of the mean-field theory for both architectures that starts from first principles by employing established methods from statistical physics of disordered systems. The theory elucidates that while the mean-field equations are different with regard to t…
Massively parallel recordings of spiking activity in cortical networks show that covariances vary widely across pairs of neurons. Their low average is well understood, but an explanation for the wide distribution in relation to the static (quenched) disorder of the connectivity in recurrent random networks was so far elusive. We here derive a finite-size mean-field theory that reduces a disordered to a highly symmetric network with fluctuating auxiliary fields. The exposed analytical relation between the statistics of connections and the statistics of pairwise covariances shows that both, aver…
Neuronal activity arises from an interaction between ongoing firing generated spontaneously by neural circuits and responses driven by external stimuli. Using mean-field analysis, we ask how a neural network that intrinsically generates chaotic patterns of activity can remain sensitive to extrinsic input. We find that inputs not only drive network responses, they also actively suppress ongoing activity, ultimately leading to a phase transition in which chaos is completely eliminated. The critical input intensity at the phase transition is a non-monotonic function of stimulus frequency, reveali…
Center for Quantitative Biology, Academy for Advanced Interdisciplinary Studies, Peking University（北京大学前沿交叉学科研究院定量生物学中心）；School of Physics, Peking University（北京大学物理学院）；Center for Bioinformatics, National Laboratory of Protein Engineering and Plant Genetic Engineering, School of Life Sciences and Center for Quantitative Biology, Academy for Advanced Interdisciplinary Studies,Peking University（北京大学生命科学学院、前沿交叉学科研究院定量生物学中心及蛋白质工程与植物基因工程国家重点实验室生物信息中心）；CNRS, LJAD and Institute NeuroMod, Université Côte d’Azur（法国蔚蓝海岸大学神经调节研究所、LJAD及法国国家科学研究中心）
We study with numerical simulation the possible limit behaviors of synchronous discrete-time deterministic recurrent neural networks composed of N binary neurons as a function of a network's level of dilution and asymmetry. The network dilution measures the fraction of neuron couples that are connected, and the network asymmetry measures to what extent the underlying connectivity matrix is asymmetric. For each given neural network, we study the dynamical evolution of all the different initial conditions, thus characterizing the full dynamical landscape without imposing any learning rule. Becau…
Random matrix theory of sparse neuronal networks with heterogeneous timescales
Complex Systems, Department of Physics, Faculty of Science, Chulalongkorn University, Bangkok, Thailand；High Energy Physics Research Unit, Faculty of Science, Chulalongkorn University, Bangkok, Thailand；International Solvay Institutes, Brussels, Belgium；Mark Kac Center for Complex Systems Research, Jagiellonian University, Krak\'ow, Poland；Department of Mathematics, King's College London, London, UK
Lecture notes: From Gaussian processes to feature learning
Institute for Advanced Simulation (IAS-6), Jülich Research Centre（先进模拟研究所）；Department of Physics, Faculty 1, RWTH Aachen University（亚琛工业大学物理系）；RWTH Aachen University（亚琛工业大学）；The Racah Institute of Physics, The Hebrew University of Jerusalem（耶路撒冷希伯来大学拉卡物理研究所）
Trial-to-trial variability is an essential feature of neural responses, but its source is a subject of active debate. Response variability (Mast and Victor, 1991; Arieli et al., 1995 & 1996; Anderson et al., 2000 & 2001; Kenet et al., 2003; Petersen et al., 2003a & b; Fiser, Chiu and Weliky, 2004; MacLean et al., 2005; Yuste et al., 2005; Vincent et al., 2007) is often treated as random noise, generated either by other brain areas, or by stochastic processes within the circuitry being studied. We call such sources of variability external to stress the independence of this form of noise from ac…
