Rich, spontaneous brain activity has been observed across a range of different temporal and spatial scales. These dynamics are thought to be important t for efficient neural functioning. Experimental evidence suggests that these neural dynamics are maintained across a variety of different cognitive states, in response to alterations of the environment and to changes in brain configuration (e.g., across individuals, development and in many neurological disorders). This suggests that the brain has evolved mechanisms to stabilize dynamics and maintain them across a range of situations. Here, we employ a local homeostatic inhibitory plasticity mechanism, balancing inhibitory and excitatory activity in a model of macroscopic brain activity based on white-matter structural connectivity. We demonstrate that the addition of homeostatic plasticity regulates network activity and allows for the emergence of rich, spontaneous dynamics across a range of brain configurations. Furthermore, the presence of homeostatic plasticity maximises the overlap between empirical and simulated patterns of functional connectivity. Therefore, this work presents a simple, local, biologically plausible inhibitory mechanism that allows stable dynamics to emerge in the brain and which facilitates the formation of functional connectivity networks.
Recently spike timing dependent plasticity was observed in inhibitory synapse in the layer II of entorhinal cortex. The rule provides an interesting zero in the region of $Δt=t_{post}-t_{pre}=0$ and in addition the dynamic range of the rule lie in gamma frequency band. We propose a robust mechanism based on this observed synaptic plasticity rule for inhibitory synapses for two mutually coupled interneurons to phase lock in synchrony in the presence of intrisic heterogeneity in firing. We study the stability of the phase locked solution by defining a map for spike times dependent on the phase r…
We are concerned about burst synchronization (BS), related to neural information processes in health and disease, in the Barabási-Albert scale-free network (SFN) composed of inhibitory bursting Hindmarsh-Rose neurons. This inhibitory neuronal population has adaptive dynamic synaptic strengths governed by the inhibitory spike-timing-dependent plasticity (iSTDP). In previous works without considering iSTDP, BS was found to appear in a range of noise intensities for fixed synaptic inhibition strengths. In contrast, in our present work, we take into consideration iSTDP and investigate its effect o…
The cortex learns to make associations between stimuli and spiking activity which supports behaviour. It does this by adjusting synaptic weights. The complexity of these transformations implies that synapses have to change without access to the full error information, a problem typically referred to as "credit-assignment". However, it remains unknown how the cortex solves this problem. We propose that a combination of plasticity rules, 1) Hebbian, 2) acetylcholine-dependent and 3) noradrenaline-dependent excitatory plasticity, together with 4) inhibitory plasticity restoring E/I balance, effec…
We investigate the dynamical role of inhibitory and highly connected nodes (hub) in synchronization and input processing of leaky-integrate-and-fire neural networks with short term synaptic plasticity. We take advantage of a heterogeneous mean-field approximation to encode the role of network structure and we tune the fraction of inhibitory neurons $f_I$ and their connectivity level to investigate the cooperation between hub features and inhibition. We show that, depending on $f_I$, highly connected inhibitory nodes strongly drive the synchronization properties of the overall network through d…
How can neural networks learn to efficiently represent complex and high-dimensional inputs via local plasticity mechanisms? Classical models of representation learning assume that input weights are learned via pairwise Hebbian-like plasticity. Here, we show that pairwise Hebbian-like plasticity only works under unrealistic requirements on neural dynamics and input statistics. To overcome these limitations, we derive from first principles a learning scheme based on voltage-dependent synaptic plasticity rules. Here, inhibition learns to locally balance excitatory input in individual dendritic co…
We consider the Watts-Strogatz small-world network (SWN) consisting of inhibitory fast spiking Izhikevich interneurons. This inhibitory neuronal population has adaptive dynamic synaptic strengths governed by the inhibitory spike-timing-dependent plasticity (iSTDP). In previous works without iSTDP, fast sparsely synchronized rhythms, associated with diverse cognitive functions, were found to appear in a range of large noise intensities for fixed strong synaptic inhibition strengths. Here, we investigate the effect of iSTDP on fast sparse synchronization (FSS) by varying the noise intensity $D$.…
The brain is believed to operate in part by making predictions about sensory stimuli and encoding deviations from these predictions in the activity of "prediction error neurons." This principle defines the widely influential theory of predictive coding. The precise circuitry and plasticity mechanisms through which animals learn to compute and update their predictions are unknown. Homeostatic inhibitory synaptic plasticity is a promising mechanism for training neuronal networks to perform predictive coding. Homeostatic plasticity causes neurons to maintain a steady, baseline firing rate in resp…
We consider clustered small-world networks with both inhibitory (I) and excitatory (E) populations. This I-E neuronal network has adaptive dynamic I to E and E to I interpopulation synaptic strengths, governed by interpopulation spike-timing-dependent plasticity (STDP). In previous works without STDPs, fast sparsely synchronized rhythms, related to diverse cognitive functions, were found to appear in a range of noise intensity $D$ for static synaptic strengths. By varying $D$, we investigate the effect of interpopulation STDPs on diverse population and individual properties of fast sparsely sy…
This paper introduces a rate-based nonlinear neural network in which excitatory (E) neurons receive feedforward excitation from sensory (S) neurons, and inhibit each other through disynaptic pathways mediated by inhibitory (I) interneurons. Correlation-based plasticity of disynaptic inhibition serves to incompletely decorrelate E neuron activity, pushing the E neurons to learn distinct sensory features. The plasticity equations additionally contain "extra" terms fostering competition between excitatory synapses converging onto the same postsynaptic neuron and inhibitory synapses diverging from…
Brain plasticity, also known as neuroplasticity, is a fundamental mechanism of neuronal adaptation in response to changes in the environment or due to brain injury. In this review, we show our results about the effects of synaptic plasticity on neuronal networks composed by Hodgkin-Huxley neurons. We show that the final topology of the evolved network depends crucially on the ratio between the strengths of the inhibitory and excitatory synapses. Excitation of the same order of inhibition revels an evolved network that presents the rich-club phenomenon, well known to exist in the brain. For ini…
The dynamics of local cortical networks are irregular, but correlated. Dynamic excitatory--inhibitory balance is a plausible mechanism that generates such irregular activity, but it remains unclear how balance is achieved and maintained in plastic neural networks. In particular, it is not fully understood how plasticity induced changes in the network affect balance, and in turn, how correlated, balanced activity impacts learning. How does the dynamics of balanced networks change under different plasticity rules? How does correlated spiking activity in recurrent networks change the evolution of…
We study the dynamics of networks with inhibitory and excitatory leaky-integrate-and-fire neurons with short-term synaptic plasticity in the presence of depressive and facilitating mechanisms. The dynamics is analyzed by a Heterogeneous Mean-Field approximation, that allows to keep track of the effects of structural disorder in the network. We describe the complex behavior of different classes of excitatory and inhibitory components, that give rise to a rich dynamical phase-diagram as a function of the fraction of inhibitory neurons. By the same mean field approach, we study and solve a global…
The aim of the present paper is to study the effects of Hebbian learning in random recurrent neural networks with biological connectivity, i.e. sparse connections and separate populations of excitatory and inhibitory neurons. We furthermore consider that the neuron dynamics may occur at a (shorter) time scale than synaptic plasticity and consider the possibility of learning rules with passive forgetting. We show that the application of such Hebbian learning leads to drastic changes in the network dynamics and structure. In particular, the learning rule contracts the norm of the weight matrix a…
We study the capacity of Hodgkin-Huxley neuron in a network to change temporarily or permanently their connections and behavior, the so called spike timing-dependent plasticity (STDP), as a function of their synchronous behavior. We consider STDP of excitatory and inhibitory synapses driven by Hebbian rules. We show that the final state of networks evolved by a STDP depend on the initial network configuration. Specifically, an initial all-to-all topology envolves to a complex topology. Moreover, external perturbations can induce co-existence of clusters, those whose neurons are synchronous and…
Precise spatio-temporal patterns of neuronal action potentials underly e.g. sensory representations and control of muscle activities. However, it is not known how the synaptic efficacies in the neuronal networks of the brain adapt such that they can reliably generate spikes at specific points in time. Existing activity-dependent plasticity rules like Spike-Timing-Dependent Plasticity are agnostic to the goal of learning spike times. On the other hand, the existing formal and supervised learning algorithms perform a temporally precise comparison of projected activity with the target, but there …
Oscillatory phase pattern formation and amplitude control for a linearized stochastic neuron field model was investigated by simulating coupled stochastic processes defined by stochastic differential equations. It was found, for several choices of parameters, that pattern formation in the phases of these processes occurred if and only if the amplitudes were allowed to grow large. Stimulated by recent work on homeostatic inhibitory plasticity, we introduced static and plastic (adaptive) systemic inhibitory mechanisms to keep the amplitudes stochastically bounded in subsequent simulations. The s…
Brain plasticity refers to brain's ability to change neuronal connections, as a result of environmental stimuli, new experiences, or damage. In this work, we study the effects of the synaptic delay on both the coupling strengths and synchronisation in a neuronal network with synaptic plasticity. We build a network of Hodgkin-Huxley neurons, where the plasticity is given by the Hebbian rules. We verify that without time delay the excitatory synapses became stronger from the high frequency to low frequency neurons and the inhibitory synapses increases in the opposite way, when the delay is incre…
Experimental and numerical results suggest that the brain can be viewed as a system acting close to a critical point, as confirmed by scale-free distributions of relevant quantities in a variety of different systems and models. Less attention has received the investigation of the temporal correlation functions in brain activity in different, healthy and pathological, conditions. Here we perform this analysis by means of a model with short and long-term plasticity which implements the novel feature of different recovery rates for excitatory and inhibitory neurons, found experimentally. We evide…
Local patterns of excitation and inhibition that can generate neural waves are studied as a computational mechanism underlying the organization of neuronal tunings. Sparse coding algorithms based on networks of excitatory and inhibitory neurons are proposed that exhibit topographic maps as the receptive fields are adapted to input stimuli. Motivated by a leaky integrate-and-fire model of neural waves, we propose an activation model that is more typical of artificial neural networks. Computational experiments with the activation model using both natural images and natural language text are pres…
Neuronal calcium (Ca2+) signaling represents a molecular trigger for diverse central nervous system adaptations and maladaptions. The altered function of dorsal spinal inhibitory interneurons is strongly implicated in the mechanisms underlying central sensitization in chronic pain. Surprisingly little is known, however, about the characteristics and consequences of Ca2+ signaling in these cells, including whether and how they are changed following a peripheral insult or injury and how such alterations might influence maladaptive pain plasticity. As a first step towards clarifying the precise r…
In recent years self organised critical neuronal models have provided insights regarding the origin of the experimentally observed avalanching behaviour of neuronal systems. It has been shown that dynamical synapses, as a form of short-term plasticity, can cause critical neuronal dynamics. Whereas long-term plasticity, such as hebbian or activity dependent plasticity, have a crucial role in shaping the network structure and endowing neural systems with learning abilities. In this work we provide a model which combines both plasticity mechanisms, acting on two different time-scales. The measure…
Understanding how recurrent neural circuits can learn to implement dynamical systems is a fundamental challenge in neuroscience. The credit assignment problem, i.e. determining the local contribution of each synapse to the network's global output error, is a major obstacle in deriving biologically plausible local learning rules. Moreover, spiking recurrent networks implementing such tasks should not be hugely costly in terms of number of neurons and spikes, as they often are when adapted from rate models. Finally, these networks should be robust to noise and neural deaths in order to sustain t…
A new brain model is introduced, based on the Impulse Pattern Formulation (IPF) already established for modeling and understanding musical instrument and rhythm perception and production. It assumes the brain works with impulses, neural bursts, ejected from an arbitrary reference point in the brain, arriving at other reflecting brain regions, and returning to the reference point delayed and damped. A plasticity model is suggested to adjust reflection strength in time. The model is systematically studied with 50 reflection points by varying the amount of excitatory vs. inhibitory neurons, the p…
Maximum Entropy models can be inferred from large data-sets to uncover how collective dynamics emerge from local interactions. Here, such models are employed to investigate neurons recorded by multielectrode arrays in the human and monkey cortex. Taking advantage of the separation of excitatory and inhibitory neuron types, we construct a model including this distinction. This approach allows to shed light upon differences between excitatory and inhibitory activity across different brain states such as wakefulness and deep sleep, in agreement with previous findings. Additionally, Maximum Entrop…
Localized persistent cortical neural activity is a validated neural substrate of parametric working memory. Such activity `bumps' represent the continuous location of a cue over several seconds. Pyramidal (excitatory) and interneuronal (inhibitory) subpopulations exhibit tuned bumps of activity, linking neural dynamics to behavioral inaccuracies observed in memory recall. However, many bump attractor models collapse these subpopulations into a single joint excitatory/inhibitory (lateral inhibitory) population, and do not consider the role of interpopulation neural architecture and noise correl…
We investigate spike-timing dependent plasticity (STPD) in the case of a synapse connecting two neural cells. We develop a theoretical analysis of several STDP rules using Markovian theory. In this context there are two different timescales, fast neural activity and slower synaptic weight updates. Exploiting this timescale separation, we derive the long-time limits of a single synaptic weight subject to STDP. We show that the pairing model of presynaptic and postsynaptic spikes controls the synaptic weight dynamics for small external input, on an excitatory synapse. This result implies in part…
Rhythmic activity in the gamma band (30-100Hz) has been observed in numerous animal species ranging from insects to humans, and in relation to a wide range of cognitive tasks. Various experimental and theoretical studies have investigated this rhythmic activity. The theoretical efforts have mainly been focused on the neuronal dynamics, under the assumption that network connectivity satisfies certain fine-tuning conditions required to generate gamma oscillations. However, it remains unclear how this fine tuning is achieved. Here we investigated the hypothesis that spike timing dependent plastic…
When inhibitory neurons constitute about 40% of neurons they could have an important antinociceptive role, as they would easily regulate the level of activity of other neurons. We consider a simple network of cortical spiking neurons with axonal conduction delays and spike timing dependent plasticity, representative of a cortical column or hypercolumn with large proportion of inhibitory neurons. Each neuron fires following a Hodgkin-Huxley like dynamics and it is interconnected randomly to other neurons. The network dynamics is investigated estimating Bandt and Pompe probability distribution f…
Continuous adaptation allows survival in an ever-changing world. Adjustments in the synaptic coupling strength between neurons are essential for this capability, setting us apart from simpler, hard-wired organisms. How these changes can be mathematically described at the phenomenological level, as so called "plasticity rules", is essential both for understanding biological information processing and for developing cognitively performant artificial systems. We suggest an automated approach for discovering biophysically plausible plasticity rules based on the definition of task families, associa…
Understanding how the brain learns to compute functions reliably, efficiently and robustly with noisy spiking activity is a fundamental challenge in neuroscience. Most sensory and motor tasks can be described as dynamical systems and could presumably be learned by adjusting connection weights in a recurrent biological neural network. However, this is greatly complicated by the credit assignment problem for learning in recurrent network, e.g. the contribution of each connection to the global output error cannot be determined based only on locally accessible quantities to the synapse. Combining …
Sequence learning, prediction and replay have been proposed to constitute the universal computations performed by the neocortex. The Hierarchical Temporal Memory (HTM) algorithm realizes these forms of computation. It learns sequences in an unsupervised and continuous manner using local learning rules, permits a context specific prediction of future sequence elements, and generates mismatch signals in case the predictions are not met. While the HTM algorithm accounts for a number of biological features such as topographic receptive fields, nonlinear dendritic processing, and sparse connectivit…
An essential step toward understanding neural circuits is linking their structure and their dynamics. In general, this relationship can be almost arbitrarily complex. Recent theoretical work has, however, begun to identify some broad principles underlying collective spiking activity in neural circuits. The first is that local features of network connectivity can be surprisingly effective in predicting global statistics of activity across a network. The second is that, for the important case of large networks with excitatory-inhibitory balance, correlated spiking persists or vanishes depending …
Critical brain hypothesis has been intensively studied both in experimental and theoretical neuroscience over the past two decades. However, some important questions still remain: (i) What is the critical point the brain operates at? (ii) What is the regulatory mechanism that brings about and maintains such a critical state? (iii) The critical state is characterized by scale-invariant behavior which is seemingly at odds with definitive brain oscillations? In this work we consider a biologically motivated model of Izhikevich neuronal network with chemical synapses interacting via spike-timingde…
Epilepsy creates a persistent increase in the probability of spontaneous seizures. An ictal episode evolves due to acute disturbance of the fine-tuned balance between excitatory vs. inhibitory inputs within a neural network in favor of excitation. The current literature that proposes the activity-dependent disinhibition as a valid mechanism of chronic epilepsy, does not provide clues on why this mechanism emerges only in epileptic patients and how the vicious circle resulting of an activity-dependent disinhibition in over-active ictogenic network would end. A new model, which presents chronic …
We consider a cerebellar ring network for the optokinetic response (OKR), and investigate the effect of diverse recoding of granule (GR) cells on OKR by varying the connection probability $p_c$ from Golgi to GR cells. For an optimal value of $p_c^*~(=0.06)$, individual GR cells exhibit diverse spiking patterns which are in-phase, anti-phase, or complex out-of-phase with respect to their population-averaged firing activity. Then, these diversely-recoded signals via parallel fibers (PFs) from GR cells are effectively depressed by the error-teaching signals via climbing fibers from the inferior o…
We investigate the efficient transmission and processing of weak, subthreshold signals in a realistic neural medium in the presence of different levels of the underlying noise. Assuming Hebbian weights for maximal synaptic conductances -- that naturally balances the network with excitatory and inhibitory synapses -- and considering short-term synaptic plasticity affecting such conductances, we found different dynamic phases in the system. This includes a memory phase where population of neurons remain synchronized, an oscillatory phase where transitions between different synchronized populatio…
Coherence resonance (CR), stochastic synchronization (SS), and spike-timing-dependent plasticity (STDP) are ubiquitous dynamical processes in biological neural networks. Whether there exists an optimal network and STDP configuration at which CR and SS are both pronounced is a fundamental question of interest that is still elusive. We expect such a configuration to enable the brain to make synergistic and optimal use of these phenomena to process information efficiently. This paper considers a small-world network of excitable Hodgkin-Huxley neurons driven by channel noise and STDP with an asymm…
Understanding of short-term synaptic depression (STSD) and other forms of synaptic plasticity is a topical problem in neuroscience. Here we study the role of STSD in the formation of complex patterns of brain rhythms. We use a cortical circuit model of neural networks composed of irregular spiking excitatory and inhibitory neurons having type 1 and 2 excitability and stochastic dynamics. In the model, neurons form a sparsely connected network and their spontaneous activity is driven by random spikes representing synaptic noise. Using simulations and analytical calculations, we found that if th…
Hebbian plasticity is a powerful principle that allows biological brains to learn from their lifetime experience. By contrast, artificial neural networks trained with backpropagation generally have fixed connection weights that do not change once training is complete. While recent methods can endow neural networks with long-term memories, Hebbian plasticity is currently not amenable to gradient descent. Here we derive analytical expressions for activity gradients in neural networks with Hebbian plastic connections. Using these expressions, we can use backpropagation to train not just the basel…
The liquid state machine (LSM) combines low training complexity and biological plausibility, which has made it an attractive machine learning framework for edge and neuromorphic computing paradigms. Originally proposed as a model of brain computation, the LSM tunes its internal weights without backpropagation of gradients, which results in lower performance compared to multi-layer neural networks. Recent findings in neuroscience suggest that astrocytes, a long-neglected non-neuronal brain cell, modulate synaptic plasticity and brain dynamics, tuning brain networks to the vicinity of the comput…
Recent studies on the complex systems have shown that the synchronization of oscillators including neuronal ones is faster, stronger, and more efficient in the small-world networks than in the regular or the random networks, and many studies are based on the assumption that the brain may utilize the small-world and scale-free network structure. We show that the functional structures in the brain are self-organized to both the small-world and the scale-free networks by synaptic re-organization by the spike timing dependent synaptic plasticity (STDP), which is hardly achieved with conventional H…
Department of Physiology University of Bern（伯尔尼大学生理学系）；Montreal Institute for Learning Algorithms Université de Montréal（蒙特利尔大学蒙特利尔学习算法研究所）
Animal behaviour depends on learning to associate sensory stimuli with the desired motor command. Understanding how the brain orchestrates the necessary synaptic modifications across different brain areas has remained a longstanding puzzle. Here, we introduce a multi-area neuronal network model in which synaptic plasticity continuously adapts the network towards a global desired output. In this model synaptic learning is driven by a local dendritic prediction error that arises from a failure to predict the top-down input given the bottom-up activities. Such errors occur at apical dendrites of …
It is widely accepted that the complex dynamics characteristic of recurrent neural circuits contributes in a fundamental manner to brain function. Progress has been slow in understanding and exploiting the computational power of recurrent dynamics for two main reasons: nonlinear recurrent networks often exhibit chaotic behavior and most known learning rules do not work in robust fashion in recurrent networks. Here we address both these problems by demonstrating how random recurrent networks (RRN) that initially exhibit chaotic dynamics can be tuned through a supervised learning rule to generat…
The error-backpropagation (backprop) algorithm remains the most common solution to the credit assignment problem in artificial neural networks. In neuroscience, it is unclear whether the brain could adopt a similar strategy to correctly modify its synapses. Recent models have attempted to bridge this gap while being consistent with a range of experimental observations. However, these models are either unable to effectively backpropagate error signals across multiple layers or require a multi-phase learning process, neither of which are reminiscent of learning in the brain. Here, we introduce a…
The brain is organized in a modular way, serving multiple functionalities. This multiplicity requires that both positive (e.g. excitatory, phase-coherent) and negative (e.g. inhibitory, phase-opposing) interactions take place across brain modules. Unfortunately, most methods to detect modules from time series either neglect or convert to positive any measured negative correlation. This may leave a significant part of the sign-dependent functional structure undetected. Here we present a novel method, based on random matrix theory, for the identification of sign-dependent modules in the brain. O…
The olfactory system removes correlations in natural odors using a network of inhibitory neurons in the olfactory bulb. It has been proposed that this network integrates the response from all olfactory receptors and inhibits them equally. However, how such global inhibition influences the neural representations of odors is unclear. Here, we study a simple statistical model of this situation, which leads to concentration-invariant, sparse representations of the odor composition. We show that the inhibition strength can be tuned to obtain sparse representations that are still useful to discrimin…
Many systems in Nature exhibit avalanche dynamics with scale-free features. A general scaling theory has been proposed for critical avalanche profiles in crackling noise, predicting the collapse onto a universal avalanche shape, as well as the scaling behaviour of the activity power spectrum as Brown noise. Recently, much attention has been given to the profile of neuronal avalanches, measured in neuronal systems in vitro and in vivo. Although a universal profile was evidenced, confirming the validity of the general scaling theory, the parallel study of the power spectrum scaling under the sam…
