An analytical description of the response properties of simple but realistic neuron models in the presence of noise is still lacking. We determine completely up to the second order the firing statistics of a single and a pair of leaky integrate-and-fire neurons (LIFs) receiving some common slowly filtered white noise. In particular, the auto- and cross-correlation functions of the output spike trains of pairs of cells are obtained from an improvement of the adiabatic approximation introduced in \cite{Mor+04}. These two functions define the firing variability and firing synchronization between neurons, and are of much importance for understanding neuron communication.
We investigate the effect of electric synapses (gap junctions) on collective neuronal dynamics and spike statistics in a conductance-based Integrate-and-Fire neural network, driven by a Brownian noise, where conductances depend upon spike history. We compute explicitly the time evolution operator and show that, given the spike-history of the network and the membrane potentials at a given time, the further dynamical evolution can be written in a closed form. We show that spike train statistics is described by a Gibbs distribution whose potential can be approximated with an explicit formula, whe…
Electrical synapses play a major role in setting up neuronal synchronization, but the precise mechanisms whereby these synapses contribute to synchrony are subtle and remain elusive. To investigate these mechanisms mean-field theories for quadratic integrate-and-fire neurons with electrical synapses have been recently put forward. Still, the validity of these theories is controversial since they assume that the neurons produce unrealistic, symmetric spikes, ignoring the well-known impact of spike shape on synchronization. Here we show that the assumption of symmetric spikes can be relaxed in s…
Conventionally, information is represented by spike rates in the neural system. Here, we consider the ability of temporally modulated activities in neuronal networks to carry information extra to spike rates. These temporal modulations, commonly known as population spikes, are due to the presence of synaptic depression in a neuronal network model. We discuss its relevance to an experiment on transparent motions in macaque monkeys by Treue et al. in 2000. They found that if the moving directions of objects are too close, the firing rate profile will be very similar to that with one direction. A…
Excitatory synaptic connections in the adult neocortex consist of multiple synaptic contacts, almost exclusively formed on dendritic spines. Changes of dendritic spine shape and volume, a correlate of synaptic strength, can be tracked in vivo for weeks. Here, we present a combined model of spike-timing dependent dendritic spine plasticity and turnover that explains the steady state multi-contact configuration of synapses in adult neocortical networks. In this model, many presynaptic neurons compete to make strong synaptic connections onto postsynaptic neurons, while the synaptic contacts compr…
A new learning scheme called time divergence-convergence (TDC) is proposed for two-layer dynamic synapse neural networks (DSNN). DSNN is an artificial neural network model, in which the synaptic transmission is modeled by a dynamic process and the information between neurons are transmitted through spike timing. In TDC, the intra-layer neurons of a DSNN are trained to map input spike trains to a higher dimension of spike trains called a feature-domain, and the output neurons are trained to build the desired spike trains by processing the spike timing of intralayer neurons. The DSNN performance…
We suggest a mechanism based on spike time dependent plasticity (STDP) of synapses to store, retrieve and predict temporal sequences. The mechanism is demonstrated in a model system of simplified integrate-and-fire type neurons densely connected by STDP synapses. All synapses are modified according to the so-called normal STDP rule observed in various real biological synapses. After conditioning through repeated input of a limited number of of temporal sequences the system is able to complete the temporal sequence upon receiving the input of a fraction of them. This is an example of effective …
Brain plasticity, also known as neuroplasticity, is a fundamental mechanism of neuronal adaptation in response to changes in the environment or due to brain injury. In this review, we show our results about the effects of synaptic plasticity on neuronal networks composed by Hodgkin-Huxley neurons. We show that the final topology of the evolved network depends crucially on the ratio between the strengths of the inhibitory and excitatory synapses. Excitation of the same order of inhibition revels an evolved network that presents the rich-club phenomenon, well known to exist in the brain. For ini…
In neural systems, synaptic plasticity is usually driven by spike trains. Due to the inherent noises of neurons, synapses and networks, spike trains typically exhibit externally uncontrollable variability such as spatial heterogeneity and temporal stochasticity, resulting in variability of synapses, which we call efficacy variability. Spike patterns with the same population rate but inducing different efficacy variability may result in neuronal networks with sharply different structures and functions. However, how the variability of spike trains influences the efficacy variability remains uncl…
Biocomputing technologies exploit biological communication mechanisms involving cell-cell signal propagation to perform computations. Researchers recently worked toward realising logic gates made by neurons to develop novel devices such as organic neuroprostheses or brain implants made by cells, herein termed living implants. Several challenges arise from this approach, mainly associated with the stochastic nature and noise of neuronal communication. Since astrocytes play a crucial role in the regulation of neurons activity, there is a possibility whereby astrocytes can be engineered to contro…
Recently spike timing dependent plasticity was observed in inhibitory synapse in the layer II of entorhinal cortex. The rule provides an interesting zero in the region of $Δt=t_{post}-t_{pre}=0$ and in addition the dynamic range of the rule lie in gamma frequency band. We propose a robust mechanism based on this observed synaptic plasticity rule for inhibitory synapses for two mutually coupled interneurons to phase lock in synchrony in the presence of intrisic heterogeneity in firing. We study the stability of the phase locked solution by defining a map for spike times dependent on the phase r…
Synfire chains are thought to underlie precisely-timed sequences of spikes observed in various brain regions and across species. How they are formed is not understood. Here we analyze self-organization of synfire chains through the spike-timing dependent plasticity (STDP) of the synapses, axon remodeling, and potentiation decay of synaptic weights in networks of neurons driven by noisy external inputs and subject to dominant feedback inhibition. Potentiation decay is the gradual, activity-independent reduction of synaptic weights over time. We show that potentiation decay enables a dynamic and…
Spiking Neural Networks (SNNs) are widely deployed to solve complex pattern recognition, function approximation and image classification tasks. With the growing size and complexity of these networks, hardware implementation becomes challenging because scaling up the size of a single array (crossbar) of fully connected neurons is no longer feasible due to strict energy budget. Modern neromorphic hardware integrates small-sized crossbars with time-multiplexed interconnects. Partitioning SNNs becomes essential in order to map them on neuromorphic hardware with the major aim to reduce the global c…
Noise-delayed decay (NDD) phenomenon emerges when the first-spike latency of a periodically forced stochastic neuron exhibits a maximum for a particular range of noise intensity. Here, we investigate the latency response dynamics of a single Hodgkin-Huxley neuron that is subject to both a suprathreshold periodic stimulus and a background activity arriving through dynamic synapses. We study the first spike latency response as a function of the presynaptic firing rate f. This constitutes a more realistic scenario than previous works, since f provides a suitable biophysically realistic parameter …
Recent evidence in rodent cerebral cortex and olfactory bulb suggests that short-term dynamics of excitatory synaptic transmission is correlated to stereotypical connectivity motifs. It was observed that neurons with short-term facilitating synapses form predominantly reciprocal pairwise connections, while neurons with short-term depressing synapses form unidirectional pairwise connections. The cause of these structural differences in synaptic microcircuits is unknown. We propose that these connectivity motifs emerge from the interactions between short-term synaptic dynamics (SD) and long-term…
We demonstrate an electrolyte-gated hybrid nanoparticle/organic synapstor (synapse-transistor, termed EGOS) that exhibits short-term plasticity as biological synapses. The response of EGOS makes it suitable to be interfaced with neurons: short-term plasticity is observed at spike voltage as low as 50 mV (in a par with the amplitude of action potential in neurons) and with a typical response time in the range of tens milliseconds. Human neuroblastoma stem cells are adhered and differentiated into neurons on top of EGOS. We observe that the presence of the cells does not alter short-term plastic…
Using a realistic model of activity dependent dynamical synapses and a standard integrate and fire neuron model we study, both analytically and numerically, the conditions in which a postsynaptic neuron efficiently detects temporal coincidences of spikes arriving at certain frequency from N different afferents. We extend a previous work that only considers synaptic depression as the most important mechanism in the transmission of information through synapses, to a more general situation including also synaptic facilitation. Our study shows that: 1) facilitation enhances the detection of correl…
Generic simulation code for spiking neuronal networks spends the major part of time in the phase where spikes have arrived at a compute node and need to be delivered to their target neurons. These spikes were emitted over the last interval between communication steps by source neurons distributed across many compute nodes and are inherently irregular with respect to their targets. For finding the targets, the spikes need to be dispatched to a three-dimensional data structure with decisions on target thread and synapse type to be made on the way. With growing network size a compute node receive…
Neurons in a micro-circuit connected by chemical synapses can have their connectivity affected by the prior activity of the cells. The number of synapses available for releasing neurotransmitter can be decreased by repetitive activation through depletion of readily releasable neurotransmitter (NT), or increased through facilitation, where the probability of release of NT is increased by prior activation. These competing effects can create a complicated and subtle range of time dependent connectivity. Here we investigate the probabilistic properties of facilitation and depression (FD) for a pre…
We study the capacity of Hodgkin-Huxley neuron in a network to change temporarily or permanently their connections and behavior, the so called spike timing-dependent plasticity (STDP), as a function of their synchronous behavior. We consider STDP of excitatory and inhibitory synapses driven by Hebbian rules. We show that the final state of networks evolved by a STDP depend on the initial network configuration. Specifically, an initial all-to-all topology envolves to a complex topology. Moreover, external perturbations can induce co-existence of clusters, those whose neurons are synchronous and…
We present a mathematical analysis of a networks with Integrate-and-Fire neurons and adaptive conductances. Taking into account the realistic fact that the spike time is only known within some \textit{finite} precision, we propose a model where spikes are effective at times multiple of a characteristic time scale $δ$, where $δ$ can be \textit{arbitrary} small (in particular, well beyond the numerical precision). We make a complete mathematical characterization of the model-dynamics and obtain the following results. The asymptotic dynamics is composed by finitely many stable periodic orbits, wh…
We study a stochastic process describing the continuous time evolution of the membrane potentials of finite system of neurons in the absence of external stimuli. The values of the membrane potentials evolve under the effect of {\it chemical synapses}, {\it electrical synapses} and a \textit{leak current}. The evolution of the process can be informally described as follows. Each neuron spikes randomly following a point process with rate depending on its membrane potential. When a neuron spikes, its membrane potential is immediately reset to a resting value. Simultaneously, the membrane potentia…
We study the spike statistics of neurons in a network with dynamically balanced excitation and inhibition. Our model, intended to represent a generic cortical column, comprises randomly connected excitatory and inhibitory leaky integrate-and-fire neurons, driven by excitatory input from an external population. The high connectivity permits a mean-field description in which synaptic currents can be treated as Gaussian noise, the mean and autocorrelation function of which are calculated self-consistently from the firing statistics of single model neurons. Within this description, we find that th…
National Engineering School of Tunis（突尼斯国立工程学院）；Université Tunis El Manar（突尼斯艾尔马纳尔大学）
Synaptic plasticity seems to be a capital aspect of the dynamics of neural networks. It is about the physiological modifications of the synapse, which have like consequence a variation of the value of the synaptic weight. The information encoding is based on the precise timing of single spike events that is based on the relative timing of the pre- and post-synaptic spikes, local synapse competitions within a single neuron and global competition via lateral connections. In order to classify temporal sequences, we present in this paper how to use a local hebbian learning, spike-timing dependent …
Reduced models of neuronal activity such as Integrate-and-Fire models allow a description of neuronal dynamics in simple, intuitive terms and are easy to simulate numerically. We present a method to fit an Integrate-and-Fire-type model of neuronal activity, namely a modified version of the Spike Response Model, to a detailed Hodgkin-Huxley-type neuron model driven by stochastic spike arrival. In the Hogkin-Huxley model, spike arrival at the synapse is modeled by a change of synaptic conductance. For such conductance spike input, more than 70% of the postsynaptic action potentials can be predic…
Spiking neurons can perform spatiotemporal feature detection by nonlinear synaptic and dendritic integration of presynaptic spike patterns. Multicompartment models of non-linear dendrites and related neuromorphic circuit designs enable faithful imitation of such dynamic integration processes, but these approaches are also associated with a relatively high computing cost or circuit size. Here, we investigate synaptic integration of spatiotemporal spike patterns with multiple dynamic synapses on point-neurons in the DYNAP-SE neuromorphic processor, which offers a complementary resource-efficient…
We investigate spike-timing dependent plasticity (STPD) in the case of a synapse connecting two neural cells. We develop a theoretical analysis of several STDP rules using Markovian theory. In this context there are two different timescales, fast neural activity and slower synaptic weight updates. Exploiting this timescale separation, we derive the long-time limits of a single synaptic weight subject to STDP. We show that the pairing model of presynaptic and postsynaptic spikes controls the synaptic weight dynamics for small external input, on an excitatory synapse. This result implies in part…
We show that a network of spiking neurons exhibits robust self-organized criticality if the synaptic efficacies follow realistic dynamics. Deriving analytical expressions for the average coupling strengths and inter-spike intervals, we demonstrate that networks with dynamical synapses exhibit critical avalanche dynamics for a wide range of interaction parameters. We prove that in the thermodynamical limit the network becomes critical for all large enough coupling parameters. We thereby explain experimental observations in which cortical neurons show avalanche activity with the total intensity …
We observe and study a self-organized phenomenon whereby the activity in a network of spiking neurons spontaneously terminates. We consider different types of populations, consisting of bistable model neurons connected electrically by gap junctions, or by either excitatory or inhibitory synapses, in a scale-free connection topology. We find that strongly synchronized population spiking events lead to complete cessation of activity in excitatory networks, but not in gap junction or inhibitory networks. We identify the underlying mechanism responsible for this phenomenon by examining the particu…
We study the synchronization of two model neurons coupled through a synapse having an activity-dependent strength. Our synapse follows the rules of Spike-Timing Dependent Plasticity (STDP). We show that this plasticity of the coupling between neurons produces enlarged frequency locking zones and results in synchronization that is more rapid and much more robust against noise than classical synchronization arising from connections with constant strength. We also present a simple discrete map model that demonstrates the generality of the phenomenon.
In timing-based neural codes, neurons have to emit action potentials at precise moments in time. We use a supervised learning paradigm to derive a synaptic update rule that optimizes via gradient ascent the likelihood of postsynaptic firing at one or several desired firing times. We find that the optimal strategy of up- and downregulating synaptic efficacies can be described by a two-phase learning window similar to that of Spike-Timing Dependent Plasticity (STDP). If the presynaptic spike arrives before the desired postsynaptic spike timing, our optimal learning rule predicts that the synapse…
We study the computational capacity of a model neuron, the Tempotron, which classifies sequences of spikes by linear-threshold operations. We use statistical mechanics and extreme value theory to derive the capacity of the system in random classification tasks. In contrast to its static analog, the Perceptron, the Tempotron's solutions space consists of a large number of small clusters of weight vectors. The capacity of the system per synapse is finite in the large size limit and weakly diverges with the stimulus duration relative to the membrane and synaptic time constants.
While spike timing has been shown to carry detailed stimulus information at the sensory periphery, its possible role in network computation is less clear. Most models of computation by neural networks are based on population firing rates. In equivalent spiking implementations, firing is assumed to be random such that averaging across populations of neurons recovers the rate-based approach. Recently, however, Denéve and colleagues have suggested that the spiking behavior of neurons may be fundamental to how neuronal networks compute, with precise spike timing determined by each neuron's contrib…
Donders Institute for Brain, Cognition and Behaviour, Radboud University（拉德堡德大学唐德斯脑、认知与行为研究所）
We propose a solution to the weight transport problem, which questions the biological plausibility of the backpropagation algorithm. We derive our method based upon a theoretical analysis of the (approximate) dynamics of leaky integrate-and-fire neurons. We show that the use of spike timing alone outcompetes existing biologically plausible methods for synaptic weight inference in spiking neural network models. Furthermore, our proposed method is more flexible, being applicable to any spiking neuron model, is conservative in how many parameters are required for implementation and can be deploye…
Measured responses from visual cortical neurons show that spike times tend to be correlated rather than exactly Poisson distributed. Fano factors vary and are usually greater than 1 due to the tendency of spikes being clustered into bursts. We show that this behavior emerges naturally in a balanced cortical network model with random connectivity and conductance-based synapses. We employ mean field theory with correctly colored noise to describe temporal correlations in the neuronal activity. Our results illuminate the connection between two independent experimental findings: high conductance s…
We present a method of estimating the gradient of an objective function with respect to the synaptic weights of a spiking neural network. The method works by measuring the fluctuations in the objective function in response to dynamic perturbation of the membrane conductances of the neurons. It is compatible with recurrent networks of conductance-based model neurons with dynamic synapses. The method can be interpreted as a biologically plausible synaptic learning rule, if the dynamic perturbations are generated by a special class of ``empiric'' synapses driven by random spike trains from an ext…
Synapses change on multiple timescales, ranging from milliseconds to minutes, due to a combination of both short- and long-term plasticity. Here we develop an extension of the common Generalized Linear Model to infer both short- and long-term changes in the coupling between a pre- and post-synaptic neuron based on observed spiking activity. We model short-term synaptic plasticity using additive effects that depend on the presynaptic spike timing, and we model long-term changes in both synaptic weight and baseline firing rate using point process adaptive smoothing. Using simulations, we first s…
Neurons are spatially extended structures that receive and process inputs on their dendrites. It is generally accepted that neuronal computations arise from the active integration of synaptic inputs along a dendrite between the input location and the location of spike generation in the axon initial segment. However, many application such as simulations of brain networks, use point-neurons --neurons without a morphological component-- as computational units to keep the conceptual complexity and computational costs low. Inevitably, these applications thus omit a fundamental property of neuronal …
The synaptic connectivity of cortical networks features an overrepresentation of certain wiring motifs compared to simple random-network models. This structure is shaped, in part, by synaptic plasticity that promotes or suppresses connections between neurons depending on their spiking activity. Frequently, theoretical studies focus on how feedforward inputs drive plasticity to create this network structure. We study the complementary scenario of self-organized structure in a recurrent network, with spike timing-dependent plasticity driven by spontaneous dynamics. We develop a self-consistent t…
Mean Field Theory Based on the Spike Time Response Curve for Synchronization Within and Between Two Alternating Populations of Neural Oscillators with Delays
Louisiana State University Health Sciences Center – New Orleans（路易斯安那州立大学健康科学中心新奥尔良分校）
Understanding of short-term synaptic depression (STSD) and other forms of synaptic plasticity is a topical problem in neuroscience. Here we study the role of STSD in the formation of complex patterns of brain rhythms. We use a cortical circuit model of neural networks composed of irregular spiking excitatory and inhibitory neurons having type 1 and 2 excitability and stochastic dynamics. In the model, neurons form a sparsely connected network and their spontaneous activity is driven by random spikes representing synaptic noise. Using simulations and analytical calculations, we found that if th…
Spike-timing dependent plasticity (STDP) is an organizing principle of biological neural networks. While synchronous firing of neurons is considered to be an important functional block in the brain, how STDP shapes neural networks possibly toward synchrony is not entirely clear. We examine relations between STDP and synchronous firing in spontaneously firing neural populations. Using coupled heterogeneous phase oscillators placed on initial networks, we show numerically that STDP prunes some synapses and promotes formation of a feedforward network. Eventually a pacemaker, which is the neuron w…
We review the use of mean field theory for describing the dynamics of dense, randomly connected cortical circuits. For a simple network of excitatory and inhibitory leaky integrate-and-fire neurons, we can show how the firing irregularity, as measured by the Fano factor, increases with the strength of the synapses in the network and with the value to which the membrane potential is reset after a spike. Generalizing the model to include conductance-based synapses gives insight into the connection between the firing statistics and the high-conductance state observed experimentally in visual cort…
Centre for Theoretical Neuroscience, University of Waterloo（滑铁卢大学理论神经科学中心）
The mathematical model underlying the Neural Engineering Framework (NEF) expresses neuronal input as a linear combination of synaptic currents. However, in biology, synapses are not perfect current sources and are thus nonlinear. Detailed synapse models are based on channel conductances instead of currents, which require independent handling of excitatory and inhibitory synapses. This, in particular, significantly affects the influence of inhibitory signals on the neuronal dynamics. In this technical report we first summarize the relevant portions of the NEF and conductance-based synapse model…
This paper presents a spike-based model which employs neurons with functionally distinct dendritic compartments for classifying high dimensional binary patterns. The synaptic inputs arriving on each dendritic subunit are nonlinearly processed before being linearly integrated at the soma, giving the neuron a capacity to perform a large number of input-output mappings. The model utilizes sparse synaptic connectivity; where each synapse takes a binary value. The optimal connection pattern of a neuron is learned by using a simple hardware-friendly, margin enhancing learning algorithm inspired by t…
HetSyn: Versatile Timescale Integration in Spiking Neural Networks via Heterogeneous Synapses
College of Intelligence and Computing, Tianjin University（智能与计算学院，天津大学）；College of Computer and Information Engineering, Tianjin Normal University（计算机与信息工程学院，天津师范大学）
An associative memory has been discussed of neural networks consisting of spiking N (=100) Hodgkin-Huxley (HH) neurons with time-delayed couplings, which memorize P patterns in their synaptic weights. In addition to excitatory synapses whose strengths are modified after the Willshaw-type learning rule with the 0/1 code for quiescent/active states, the network includes uniform inhibitory synapses which are introduced to reduce cross-talk noises. Our simulations of the HH neuron network for the noise-free state have shown to yield a fairly good performance with the storage capacity of $α_c = P_{…
This paper presents a constructive algorithm that achieves successful one-shot learning of hidden spike-patterns in a competitive detection task. It has previously been shown (Masquelier et al., 2008) that spike-timing-dependent plasticity (STDP) and lateral inhibition can result in neurons competitively tuned to repeating spike-patterns concealed in high rates of overall presynaptic activity. One-shot construction of neurons with synapse weights calculated as estimates of converged STDP outcomes results in immediate selective detection of hidden spike-patterns. The capability of continual lea…
A large effort is devoted to the research of new computing paradigms associated to innovative nanotechnologies that should complement and/or propose alternative solutions to the classical Von Neumann/CMOS association. Among various propositions, Spiking Neural Network (SNN) seems a valid candidate. (i) In terms of functions, SNN using relative spike timing for information coding are deemed to be the most effective at taking inspiration from the brain to allow fast and efficient processing of information for complex tasks in recognition or classification. (ii) In terms of technology, SNN may be…
By recording multiple cells simultaneously, electrophysiologists have found evidence for repeating spatiotemporal spike patterns, which can carry information. How this information is extracted by downstream neurons is unclear. In this theoretical paper, we investigate to what extent a single cell could detect a given spike pattern and what the optimal parameters to do so are, in particular the membrane time constant $τ$. Using a leaky integrate-and-fire (LIF) neuron with instantaneous synapses and homogeneous Poisson input, we were able to compute this optimum analytically. Our results indicat…
In spiking neural networks, the information is conveyed by the spike times, that depend on the intrinsic dynamics of each neuron, the input they receive and on the connections between neurons. In this article we study the Markovian nature of the sequence of spike times in stochastic neural networks, and in particular the ability to deduce from a spike train the next spike time, and therefore produce a description of the network activity only based on the spike times regardless of the membrane potential process. To study this question in a rigorous manner, we introduce and study an event-based …
Nanomagnets driven by spin currents provide a natural implementation for a neuron and a synapse: currents allow convenient summation of multiple inputs, while the magnet provides the threshold function. The objective of this paper is to explore the possibility of a hardware neural network (HNN) implementation using a spin switch (SS) as its basic building block. SS is a recently proposed device based on established technology with a transistor-like gain and input-output isolation. This allows neural networks to be constructed with purely passive interconnections without intervening clocks or a…
