Two recent experimental observations pose a challenge to many cortical models. First, the activity in the auditory cortex is sparse, and firing rates can be described by a lognormal distribution. Second, the distribution of non-zero synaptic strengths between nearby cortical neurons can also be described by a lognormal distribution. Here we use a simple model of cortical activity to reconcile these observations. The model makes the experimentally testable prediction that synaptic efficacies onto a given cortical neuron are statistically correlated, i.e. it predicts that some neurons receive many more strong connections than other neurons. We propose a simple Hebb-like learning rule which gives rise to both lognormal firing rates and synaptic efficacies. Our results represent a first step toward reconciling sparse activity and sparse connectivity in cortical networks.
Correlation-based Hebbian plasticity is thought to shape neuronal connectivity during development and learning, whereas homeostatic plasticity would stabilize network activity. Here we investigate another, new aspect of this dichotomy: Can Hebbian associative properties also emerge as a network effect from a plasticity rule based on homeostatic principles on the neuronal level? To address this question, we simulated a recurrent network of leaky integrate-and-fire neurons, in which excitatory connections are subject to a structural plasticity rule based on firing rate homeostasis. We show that …
A spontaneously active neural system that is capable of continual learning should also be capable of homeostasis of both firing rate and connectivity. Experimental evidence suggests that both types of homeostasis exist, and that connectivity is maintained at a state that is optimal for information transmission and storage. This state is referred to as the critical state. We present a simple stochastic computational Hebbian learning model that incorporates both firing rate and critical homeostasis, and we explore its stability and connectivity properties. We also examine the behavior of our mod…
Heterogeneity of firing rate statistics is known to have severe consequences on neural coding. Recent experimental recordings in weakly electric fish indicate that the distribution-width of superficial pyramidal cell firing rates (trial- and time-averaged) in the electrosensory lateral line lobe (ELL) depends on the stimulus, and also that network inputs can mediate changes in the firing rate distribution across the population. We previously developed theoretical methods to understand how two attributes (synaptic and intrinsic heterogeneity) interact and alter the firing rate distribution in a…
Spiking neural networks (SNNs) communicate through the all-or-none spiking activity of neurons. However, fitting the large number of SNN model parameters to observed neural activity patterns, for example, in biological experiments, remains a challenge. Previous work using genetic algorithm (GA) optimisation on a specific efficient SNN model, using the Izhikevich neuronal model, was limited to a single parameter and objective. This work applied a version of GA, called non-dominated sorting GA (NSGA-III), to demonstrate the feasibility of performing multi-objective optimisation on the same SNN, …
We present a complete mean field theory for a balanced state of a simple model of an orientation hypercolumn. The theory is complemented by a description of a numerical procedure for solving the mean-field equations quantitatively. With our treatment, we can determine self-consistently both the firing rates and the firing correlations, without being restricted to specific neuron models. Here, we solve the analytically derived mean-field equations numerically for integrate-and-fire neurons. Several known key properties of orientation selective cortical neurons emerge naturally from the descript…
Neuronal networks are controlled by a combination of the dynamics of individual neurons and the connectivity of the network that links them together. We study a minimal model of the preBotzinger complex, a small neuronal network that controls the breathing rhythm of mammals through periodic firing bursts. We show that the properties of a such a randomly connected network of identical excitatory neurons are fundamentally different from those of uniformly connected neuronal networks as described by mean-field theory. We show that (i) the connectivity properties of the networks determines the loc…
In the past decade, the cell-type specific connectivity and activity of local cortical networks have been characterized experimentally to some detail. In parallel, modeling has been established as a tool to relate network structure to activity dynamics. While the available connectivity maps have been used in various computational studies, prominent features of the simulated activity such as the spontaneous firing rates do not match the experimental findings. Here, we show that the inconsistency arises from the incompleteness of the connectivity maps. Our comparison of the most comprehensive ma…
Research on network mechanisms and coding properties of grid cells assume that the firing rate of a grid cell in each of its fields is the same. Furthermore, proposed network models predict spatial regularities in the firing of inhibitory interneurons that are inconsistent with experimental data. In this paper, by analyzing the response of grid cells recorded from rats during free navigation, we first show that there are strong variations in the mean firing rate of the fields of individual grid cells and thus show that the data is inconsistent with the theoretical models that predict similar p…
We study the spike statistics of neurons in a network with dynamically balanced excitation and inhibition. Our model, intended to represent a generic cortical column, comprises randomly connected excitatory and inhibitory leaky integrate-and-fire neurons, driven by excitatory input from an external population. The high connectivity permits a mean-field description in which synaptic currents can be treated as Gaussian noise, the mean and autocorrelation function of which are calculated self-consistently from the firing statistics of single model neurons. Within this description, we find that th…
We propose a model for description of activity-dependent evolution and self-wiring between binary neurons. Specifically, this model can be used for investigation of growth of neuronal connectivity in the developing neocortex. By using computational simulations with appropriate training pattern sequences, we show that long-term memory can be encoded in neuronal connectivity and that the external stimulations form part of the functioning neocortical circuit. It is proposed that such binary neuron representations of point-like cortical neurons fail to reproduce cytoarchitectural differences of th…
The experimental study of neural networks requires simultaneous measurements of a massive number of neurons, while monitoring properties of the connectivity, synaptic strengths and delays. Current technological barriers make such a mission unachievable. In addition, as a result of the enormous number of required measurements, the estimated network parameters would differ from the original ones. Here we present a versatile experimental technique, which enables the study of recurrent neural networks activity while being capable of dictating the network connectivity and synaptic strengths. This m…
Population rate or activity equations are the foundation of a common approach to modeling for neural networks. These equations provide mean field dynamics for the firing rate or activity of neurons within a network given some connectivity. The shortcoming of these equations is that they take into account only the average firing rate while leaving out higher order statistics like correlations between firing. A stochastic theory of neural networks which includes statistics at all orders was recently formulated. We describe how this theory yields a systematic extension to population rate equation…
Networks of randomly connected neurons are among the most popular models in theoretical neuroscience. The connectivity between neurons in the cortex is however not fully random, the simplest and most prominent deviation from randomness found in experimental data being the overrepresentation of bidirectional connections among pyramidal cells. Using numerical and analytical methods, we investigated the effects of partially symmetric connectivity on dynamics in networks of rate units. We considered the two dynamical regimes exhibited by random neural networks: the weak-coupling regime, where the …
Recent experimental and computational evidence suggests that several dynamical properties may characterize the operating point of functioning neural networks: critical branching, neutral stability, and production of a wide range of firing patterns. We seek the simplest setting in which these properties emerge, clarifying their origin and relationship in random, feedforward networks of McCullochs-Pitts neurons. Two key parameters are the thresholds at which neurons fire spikes, and the overall level of feedforward connectivity. When neurons have low thresholds, we show that there is always a co…
We study the statistics of spike trains of simultaneously recorded grid cells in freely behaving rats. We evaluate pairwise correlations between these cells and, using a generalized linear model (kinetic Ising model), study their functional connectivity. Even when we account for the covariations in firing rates due to overlapping fields, both the pairwise correlations and functional connections decay as a function of the shortest distance between the vertices of the spatial firing pattern of pairs of grid cells, i.e. their phase difference. The functional connectivity takes positive values bet…
Functions of brain areas in complex animals are believed to rely on the dynamics of networks of neurons rather than on single neurons. On the other hand, the network dynamics reflect and arise from the integration and coordination of the activity of populations of single neurons. Understanding how single-neurons and neural-circuits dynamics complement each other to produce brain functions is thus of paramount importance. LFPs and EEGs are good indicators of the dynamics of mesoscopic and macroscopic populations of neurons, while microscopic-level activities can be documented by measuring the m…
The time elapsed model describes the firing activity of an homogeneous assembly of neurons thanks to the distribution of times elapsed since the last discharge. It gives a mathematical description of the probability density of neurons structured by this time. In an earlier work, based on generalized relative entropy methods, it is proved that for highly or weakly connected networks the model exhibits relaxation to the steady state and for moderately connected networks it is obtained numerical evidence of appearance of self-sustained periodic solutions. Here, we go further and, using the partic…
Our current understanding of neuroplasticity paints a picture of a complex interconnected system of dependent processes which shape cortical structure so as to produce an efficient information processing system. Indeed, the cooperation of these processes is associated with robust, stable, adaptable networks with characteristic features of activity and synaptic topology. However, combining the actions of these mechanisms in models has proven exceptionally difficult and to date no model has been able to do so without significant hand-tuning. Until such a model exists that can successfully combin…
Firing patterns in the central nervous system often exhibit strong temporal irregularity and heterogeneity in their time averaged response properties. Previous studies suggested that these properties are outcome of an intrinsic chaotic dynamics. Indeed, simplified rate-based large neuronal networks with random synaptic connections are known to exhibit sharp transition from fixed point to chaotic dynamics when the synaptic gain is increased. However, the existence of a similar transition in neuronal circuit models with more realistic architectures and firing dynamics has not been established. I…
Neurons in the primary visual cortex are more or less selective for the orientation of a light bar used for stimulation. A broad distribution of individual grades of orientation selectivity has in fact been reported in all species. A possible reason for emergence of broad distributions is the recurrent network within which the stimulus is being processed. Here we compute the distribution of orientation selectivity in randomly connected model networks that are equipped with different spatial patterns of connectivity. We show that, for a wide variety of connectivity patterns, a linear theory bas…
In this note, we develop semi-analytical techniques to obtain the full correlational structure of a stochastic network of nonlinear neurons described by rate variables. Under the assumption that pairs of membrane potentials are jointly Gaussian -- which they tend to be in large networks -- we obtain deterministic equations for the temporal evolution of the mean firing rates and the noise covariance matrix that can be solved straightforwardly given the network connectivity. We also obtain spike count statistics such as Fano factors and pairwise correlations, assuming doubly-stochastic action po…
The representation of the natural-density, heterogeneous connectivity of neuronal network models at relevant spatial scales remains a challenge for Computational Neuroscience and Neuromorphic Computing. In particular, the memory demands imposed by the vast number of synapses in brain-scale network simulations constitutes a major obstacle. Limiting the number resolution of synaptic weights appears to be a natural strategy to reduce memory and compute load. In this study, we investigate the effects of a limited synaptic-weight resolution on the dynamics of recurrent spiking neuronal networks res…
The nonlinear noisy leaky integrate-and-fire (NNLIF) model is a popular mean-field description of a large number of interacting neurons, which has attracted mathematicians to study from various aspects. A core property of this model is the finite time blow-up of the firing rate, which scientifically corresponds to the synchronization of a neuron network, and mathematically prevents the existence of a global classical solution. In this work, we propose a new generalized solution based on reformulating the PDE model with a specific change of variable in time. A firing rate dependent timescale is…
Oscillations are omnipresent in neural population signals, like multi-unit recordings, EEG/MEG, and the local field potential. They have been linked to the population firing rate of neurons, with individual neurons firing in a close-to-irregular fashion at low rates. Using a combination of mean-field and linear response theory we predict the spectra generated in a layered microcircuit model of V1, composed of leaky integrate-and-fire neurons and based on connectivity compiled from anatomical and electrophysiological studies. The model exhibits low- and high-gamma oscillations visible in all po…
Recurrent networks of non-linear units display a variety of dynamical regimes depending on the structure of their synaptic connectivity. A particularly remarkable phenomenon is the appearance of strongly fluctuating, chaotic activity in networks of deterministic, but randomly connected rate units. How this type of intrinsi- cally generated fluctuations appears in more realistic networks of spiking neurons has been a long standing question. To ease the comparison between rate and spiking networks, recent works investigated the dynami- cal regimes of randomly-connected rate networks with segrega…
Synaptic plasticity and neuron cross-talk are some of the important key mechanisms underlying formation of dynamic clusters of active neurons. The essence of this study is to model and decipher the mechanism of emergence of a task-specific functional connectivity among a population of neurons. We have used our proposed neuron activity pattern (NAP) model to define an assorted range of interactions and simulated across a 2D lattice with two-state (firing/non-firing) neurons. We observed the emergence of scale-free, hierarchically organized ordered patterns of active neurons, that we call as fun…
The synaptic connectivity of cortex is plastic, with experience shaping the ongoing interactions between neurons. Theoretical studies of spike timing-dependent plasticity (STDP) have focused on either just pairs of neurons or large-scale simulations where analytic insight is lacking. A simple account for how fast spike time correlations affect both micro- and macroscopic network structure remains lacking. We develop a low-dimensional mean field theory showing how STDP gives rise to strongly coupled assemblies of neurons with shared stimulus preferences, with the connectivity actively reinforce…
Nonlinear Noisy Leaky Integrate and Fire (NNLIF) models for neurons networks can be written as Fokker-Planck-Kolmogorov equations on the probability density of neurons, the main parameters in the model being the connectivity of the network and the noise. We analyse several aspects of the NNLIF model: the number of steady states, a priori estimates, blow-up issues and convergence toward equilibrium in the linear case. In particular, for excitatory networks, blow-up always occurs for initial data concentrated close to the firing potential. These results show how critical is the balance between n…
This paper presents a Bayesian approach to learning the connectivity structure of a group of neurons from data on configuration frequencies. A major objective of the research is to provide statistical tools for detecting changes in firing patterns with changing stimuli. Our framework is not restricted to the well-understood case of pair interactions, but generalizes the Boltzmann machine model to allow for higher order interactions. The paper applies a Markov Chain Monte Carlo Model Composition (MC3) algorithm to search over connectivity structures and uses Laplace's method to approximate post…
Prior to receiving visual stimuli, spontaneous, correlated activity called retinal waves drives activity-dependent developmental programs. Early-stage waves mediated by acetylcholine (ACh) manifest as slow, spreading bursts of action potentials. They are believed to be initiated by the spontaneous firing of Starburst Amacrine Cells (SACs), whose dense, recurrent connectivity then propagates this activity laterally. Their extended inter-wave intervals and shifting wave boundaries are the result of the slow after-hyperpolarization of the SACs creating an evolving mosaic of recruitable and refrac…
A method of data assimilation (DA) is employed to estimate electrophysiological parameters of neurons simultaneously with their synaptic connectivity in a small model biological network. The DA procedure is cast as an optimization, with a cost function consisting of both a measurement error and a model error term. An iterative reweighting of these terms permits a systematic method to identify the lowest minimum, within a local region of state space, on the surface of a non-convex cost function. In the model, two sets of parameter values are associated with two particular functional modes of ne…
Here we demonstrate that the activity of neural ensembles can be quantitatively modeled. We first show that an ensemble dynamical model (EDM) accurately approximates the distribution of voltages and average firing rate per neuron of a population of simulated integrate-and-fire neurons. EDMs are high-dimensional nonlinear dynamical models. To faciliate the estimation of their parameters we present a dimensionality reduction method and study its performance with simulated data. We then introduce and evaluate a maximum-likelihood method to estimate connectivity parameters in networks of EDMS. Fin…
Networks of living neurons exhibit an avalanche mode of activity, experimentally found in organotypic cultures. Moreover, experimental studies of morphology indicate that neurons develop a network of small-world-like connections, with the possibility of very high connectivity degree. Here we study a recent model based on self-organized criticality, which consists of an electrical network with threshold firing and activity-dependent synapse strengths. We study the model on a scale-free network, the Apollonian network, which presents many features of neuronal systems. The system exhibits a power…
Neuronal dynamics are fundamentally constrained by the underlying structural network architecture, yet much of the details of this synaptic connectivity are still unknown even in neuronal cultures in vitro. Here we extend a previous approach based on information theory, the Generalized Transfer Entropy, to the reconstruction of connectivity of simulated neuronal networks of both excitatory and inhibitory neurons. We show that, due to the model-free nature of the developed measure, both kinds of connections can be reliably inferred if the average firing rate between synchronous burst events exc…
The oculomotor delayed-response (ODR) task is a common experimental paradigm of working memory (WM) study, in which a monkey must fixate its gaze on the center of a screen and, following a brief cue that flashes on the screen, keep fixating for several more seconds before shifting its gaze to the location where the cue flashed. Consequently, in the delay period between the cue and the response the monkey must maintain a memory of cue location. Electrophysiological recordings from the prefrontal area of the cortex (PFC) revealed neurons that display selective persistent activity: their firing r…
Unraveling the interplay between connectivity and spatio-temporal dynamics in neuronal networks is a key step to advance our understanding of neuronal information processing. Here we investigate how particular features of network connectivity underpin the propensity of neural networks to generate slow-switching assembly (SSA) dynamics, i.e., sustained epochs of increased firing within assemblies of neurons which transition slowly between different assemblies throughout the network. We show that the emergence of SSA activity is linked to spectral properties of the asymmetric synaptic weight mat…
How spiking activity reverberates through neuronal networks, how evoked and spontaneous activity interact and blend, and how the combined activities represent external stimulation are pivotal questions in neuroscience. We simulated minimal models of unstructured spiking networks in silico, asking whether and how gentle external stimulation might be subsequently reflected in spontaneous activity fluctuations. Consistent with earlier findings in silico and in vitro, we observe a privileged sub-population of 'pioneer neurons' that, by their firing order, reliably encode previous external stimulat…
Laboratory of Biological Modeling, NIDDK/NIH（美国国立糖尿病、消化系统疾病和肾病研究所（NIDDK）/美国国立卫生研究院（NIH）生物建模实验室）
Spiking activity of neurons engaged in learning and performing a task show complex spatiotemporal dynamics. While the output of recurrent network models can learn to perform various tasks, the possible range of recurrent dynamics that emerge after learning remains unknown. Here we show that modifying the recurrent connectivity with a recursive least squares algorithm provides sufficient flexibility for synaptic and spiking rate dynamics of spiking networks to produce a wide range of spatiotemporal activity. We apply the training method to learn arbitrary firing patterns, stabilize irregular sp…
Cortical synapse organization supports a range of dynamic states on multiple spatial and temporal scales, from synchronous slow wave activity (SWA), characteristic of deep sleep or anesthesia, to fluctuating, asynchronous activity during wakefulness (AW). Such dynamic diversity poses a challenge for producing efficient large-scale simulations that embody realistic metaphors of short- and long-range synaptic connectivity. In fact, during SWA and AW different spatial extents of the cortical tissue are active in a given timespan and at different firing rates, which implies a wide variety of loads…
The dynamics of neural networks is often characterized by collective behavior and quasi-synchronous events, where a large fraction of neurons fire in short time intervals, separated by uncorrelated firing activity. These global temporal signals are crucial for brain functioning. They strongly depend on the topology of the network and on the fluctuations of the connectivity. We propose a heterogeneous mean--field approach to neural dynamics on random networks, that explicitly preserves the disorder in the topology at growing network sizes, and leads to a set of self-consistent equations. Within…
Nonlinear dynamics of random neural networks with second-order synaptic motifs
Interdisciplinary Graduate Program in Quantitative Biosciences, Georgia Institute of Technology（佐治亚理工学院定量生物科学跨学科研究生项目）；School of Mathematics, Georgia Institute of Technology（佐治亚理工学院数学学院）
