In this article, our wish is to demystify some aspects of coding with spike-timing, through a simple review of well-understood technical facts regarding spike coding. The goal is to help better understanding to which extend computing and modelling with spiking neuron networks can be biologically plausible and computationally efficient. We intentionally restrict ourselves to a deterministic dynamics, in this review, and we consider that the dynamics of the network is defined by a non-stochastic mapping. This allows us to stay in a rather simple framework and to propose a review with concrete numerical values, results and formula on (i) general time constraints, (ii) links between continuous signals and spike trains, (iii) spiking networks parameter adjustments. When implementing spiking neuron networks, for computational or biological simulation purposes, it is important to take into account the indisputable facts here reviewed. This precaution could prevent from implementing mechanisms meaningless with regards to obvious time constraints, or from introducing spikes artificially, when continuous calculations would be sufficient and simpler. It is also pointed out that implementing a spiking neuron network is finally a simple task, unless complex neural codes are considered.
We establish a general linear response relation for spiking neuronal networks, based on chains with unbounded memory. This relation allows us to predict the influence of a weak amplitude time-dependent external stimuli on spatio-temporal spike correlations, from the spontaneous statistics (without stimulus) in a general context where the memory in spike dynamics can extend arbitrarily far in the past. Using this approach, we show how linear response is explicitly related to neuronal dynamics with an example, the gIF model, introduced by M. Rudolph and A. Destexhe. This example illustrates the …
Spike correlations between neurons are ubiquitous in the cortex, but their role is at present not understood. Here we describe the firing response of a leaky integrate-and-fire neuron (LIF) when it receives a temporarily correlated input generated by presynaptic correlated neuronal populations. Input correlations are characterized in terms of the firing rates, Fano factors, correlation coefficients and correlation timescale of the neurons driving the target neuron. We show that the sum of the presynaptic spike trains cannot be well described by a Poisson process. Solutions of the output firing…
We are interested in characterization of synchronization transitions of bursting neurons in the frequency domain. Instantaneous population firing rate (IPFR) $R(t)$, which is directly obtained from the raster plot of neural spikes, is often used as a realistic collective quantity describing population activities in both the computational and the experimental neuroscience. For the case of spiking neurons, a realistic time-domain order parameter, based on $R(t)$, was introduced in our recent work to characterize the spike synchronization transition. Unlike the case of spiking neurons, the IPFR $…
We numerically investigate the influence of intrinsic channel noise on the dynamical response of delay-coupling in neuronal systems. The stochastic dynamics of the spiking is modeled within a stochastic modification of the standard Hodgkin-Huxley model wherein the delay-coupling accounts for the finite propagation time of an action potential along the neuronal axon. We quantify this delay-coupling of the Pyragas-type in terms of the difference between corresponding presynaptic and postsynaptic membrane potentials. For an elementary neuronal network consisting of two coupled neurons we detect c…
The effect of environmental temperature on neuronal spiking behaviors is investigated by numerically simulating the temperature dependence of spiking threshold of the Hodgkin-Huxley neuron subject to synaptic stimulus. We find that the spiking threshold exhibits a global minimum in a "comfortable temperature" range where spike initiation needs weakest synaptic strength, indicating the occurrence of optimal use of synaptic transmission in neural system. We further explore the biophysical origin of this phenomenon in ion channel gating kinetics and also discuss its possible biological relevance …
Recurrent neural networks are powerful tools for understanding and modeling computation and representation by populations of neurons. Continuous-variable or "rate" model networks have been analyzed and applied extensively for these purposes. However, neurons fire action potentials, and the discrete nature of spiking is an important feature of neural circuit dynamics. Despite significant advances, training recurrently connected spiking neural networks remains a challenge. We present a procedure for training recurrently connected spiking networks to generate dynamical patterns autonomously, to p…
Humans and other animals behave as if we perform fast Bayesian inference underlying decisions and movement control given uncertain sense data. Here we show that a biophysically realistic model of the subthreshold membrane potential of a single neuron can exactly compute the numerator in Bayes rule for inferring the Poisson parameter of a sensory spike train. A simple network of spiking neurons can construct and represent the Bayesian posterior density of a parameter of an external cause that affects the Poisson parameter, accurately and in real time.
We investigate the regularity of spontaneous spiking activity on Newman-Watts small-world networks consisting of biophysically realistic Hodgkin-Huxley neurons with a tunable intensity of intrinsic noise and fraction of blocked voltage-gated sodium and potassium ion channels embedded in neuronal membranes. We show that there exists an optimal fraction of shortcut links between physically distant neurons, as well as an optimal intensity of intrinsic noise, which warrant an optimally ordered spontaneous spiking activity. This doubly coherence resonance-like phenomenon depends significantly, and …
The response of a neuron to synaptic input strongly depends on whether or not it has just emitted a spike. We propose a neuron model that after spike emission exhibits a partial response to residual input charges and study its collective network dynamics analytically. We uncover a novel desynchronization mechanism that causes a sequential desynchronization transition: In globally coupled neurons an increase in the strength of the partial response induces a sequence of bifurcations from states with large clusters of synchronously firing neurons, through states with smaller clusters to completel…
In this article, we study optimal control problems of spiking neurons whose dynamics are described by a phase model. We design minimum-power current stimuli (controls) that lead to targeted spiking times of neurons, where the cases with unbounded and bounded control amplitude are considered. We show that theoretically the spiking period of a neuron, modeled by phase dynamics, can be arbitrarily altered by a smooth control. However, if the control amplitude is bounded, the range of possible spiking times is constrained and determined by the bound, and feasible spiking times are optimally achiev…
We consider a wide class of spiking neuron models, defined by rather general set of conditions typical for basic models like leaky integrate and fire, or binding neuron model. A neuron is fed with a point renewal process. A relation between the three probability density functions (pdf): (i) pdf of input interspike intervals, (ii) pdf of output interspike intervals of a neuron with instantaneous feedback and (iii) pdf for that same neuron without feedback is derived. This allows to calculate any of the three pdfs provided the another two are given. Similar relation between corresponding means a…
We study the effect of intrinsic heterogeneity on the activity of a population of leaky integrate-and-fire neurons. By rescaling the dynamical equation, we derive mathematical relations between multiple neuronal parameters and a fluctuating input noise. To this end, common input to heterogeneous neurons is conceived as an identical noise with neuron-specific mean and variance. As a consequence, the neuronal output rates can differ considerably, and their relative spike timing becomes desynchronized. This theory can quantitatively explain some recent experimental findings.
Finite-sized populations of spiking elements are fundamental to brain function, but also used in many areas of physics. Here we present a theory of the dynamics of finite-sized populations of spiking units, based on a quasi-renewal description of neurons with adaptation. We derive an integral equation with colored noise that governs the stochastic dynamics of the population activity in response to time-dependent stimulation and calculate the spectral density in the asynchronous state. We show that systems of coupled populations with adaptation can generate a frequency band in which sensory inf…
Networks of model neurons with balanced recurrent excitation and inhibition produce irregular and asynchronous spiking activity. We extend the analysis of balanced networks to include the known dependence of connection probability on the spatial separation between neurons. In the continuum limit we derive that stable, balanced firing rate solutions require that the spatial spread of external inputs be broader than that of recurrent excitation, which in turn must be broader than or equal to that of recurrent inhibition. For finite size networks we investigate the pattern forming dynamics arisin…
A long tradition in theoretical neuroscience casts sensory processing in the brain as the process of inferring the maximally consistent interpretations of imperfect sensory input. Recently it has been shown that Gamma-band inhibition can enable neural attractor networks to approximately carry out such a sampling mechanism. In this paper we propose a novel neural network model based on irregular gating inhibition, show analytically how it implements a Monte-Carlo Markov Chain (MCMC) sampler, and describe how it can be used to model networks of both neural attractors as well as of single spiking…
Spiking networks that perform probabilistic inference have been proposed both as models of cortical computation and as candidates for solving problems in machine learning. However, the evidence for spike-based computation being in any way superior to non-spiking alternatives remains scarce. We propose that short-term plasticity can provide spiking networks with distinct computational advantages compared to their classical counterparts. In this work, we use networks of leaky integrate-and-fire neurons that are trained to perform both discriminative and generative tasks in their forward and back…
Many central neurons, and in particular certain brainstem aminergic neurons exhibit spontaneous and fairly regular spiking with frequencies of order a few Hz. A large number of ion channel types contribute to such spiking so that accurate modeling of spike generation leads to the requirement of solving very large systems of differential equations, ordinary in the first instance. Since analysis of spiking behavior when many synaptic inputs are active adds further to the number of components, it is useful to have simplified mathematical models of spiking in such neurons so that, for example, sto…
At functional scales, cortical behavior results from the complex interplay of a large number of excitable cells operating in noisy environments. Such systems resist to mathematical analysis, and computational neurosciences have largely relied on heuristic partial (and partially justified) macroscopic models, which successfully reproduced a number of relevant phenomena. The relationship between these macroscopic models and the spiking noisy dynamics of the underlying cells has since then been a great endeavor. Based on recent mean-field reductions for such spiking neurons, we present here {a pr…
Limitations in processing capabilities and memory of today's computers make spiking neuron-based (human) whole-brain simulations inevitably characterized by a compromise between bio-plausibility and computational cost. It translates into brain models composed of a reduced number of neurons and a simplified neuron's mathematical model, leading to the search for new simulation strategies. Taking advantage of the sparse character of brain-like computation, the event-driven technique could represent a way to carry out efficient simulation of large-scale Spiking Neural Networks (SNN). The recent Le…
We are interested in characterization of population synchronization of bursting neurons which exhibit both the slow bursting and the fast spiking timescales, in contrast to spiking neurons. Population synchronization may be well visualized in the raster plot of neural spikes which can be obtained in experiments. The instantaneous population firing rate (IPFR) $R(t)$, which may be directly obtained from the raster plot of spikes, is often used as a realistic collective quantity describing population behaviors in both the computational and the experimental neuroscience. For the case of spiking n…
Several studies with brain signals suggested that bottom-up and top-down influences are exerted through distinct frequency bands among visual cortical areas. It has been recently shown that theta and gamma rhythms subserve feedforward, whereas the feedback influence is dominated by the alpha-beta rhythm in primates. A few theoretical models for reproducing these effects have been proposed so far. Here we show that a simple but biophysically plausible two-network motif composed of spiking-neuron models and chemical synapses can exhibit feedforward and feedback influences through distinct freque…
We propose a model of an adaptive network of spiking neurons that gives rise to a hypernetwork of its dynamic states at the upper level of description. Left to itself, the network exhibits a sequence of transient clustering which relates to a traffic in the hypernetwork in the form of a random walk. Receiving inputs the system is able to generate reproducible sequences corresponding to stimulus-specific paths in the hypernetwork. We illustrate these basic notions by a simple network of discrete-time spiking neurons together with its FPGA realization and analyze their properties.
We provide rigorous and exact results characterizing the statistics of spike trains in a network of leaky integrate and fire neurons, where time is discrete and where neurons are submitted to noise, without restriction on the synaptic weights. We show the existence and uniqueness of an invariant measure of Gibbs type and discuss its properties. We also discuss Markovian approximations and relate them to the approaches currently used in computational neuroscience to analyse experimental spike trains statistics.
Spiking neural network is a type of artificial neural network in which neurons communicate between each other with spikes. Spikes are identical Boolean events characterized by the time of their arrival. A spiking neuron has internal dynamics and responds to the history of inputs as opposed to the current inputs only. Because of such properties a spiking neural network has rich intrinsic capabilities to process spatiotemporal data. However, because the spikes are discontinuous 'yes or no' events, it is not trivial to apply traditional training procedures such as gradient descend to the spiking …
Top-down feedback in cortex is critical for guiding sensory processing, which has prominently been formalized in the theory of hierarchical predictive coding (hPC). However, experimental evidence for error units, which are central to the theory, is inconclusive, and it remains unclear how hPC can be implemented with spiking neurons. To address this, we connect hPC to existing work on efficient coding in balanced networks with lateral inhibition, and predictive computation at apical dendrites. Together, this work points to an efficient implementation of hPC with spiking neurons, where predictio…
We demonstrate a spiking neural network for navigation motivated by the chemotaxis network of Caenorhabditis elegans. Our network uses information regarding temporal gradients in the tracking variable's concentration to make navigational decisions. The gradient information is determined by mimicking the underlying mechanisms of the ASE neurons of C. elegans. Simulations show that our model is able to forage and track a target set-point in extremely noisy environments. We develop a VLSI implementation for the main gradient detector neurons, which could be integrated with standard comparator cir…
The seemingly stochastic transient dynamics of neocortical circuits observed in vivo have been hypothesized to represent a signature of ongoing stochastic inference. In vitro neurons, on the other hand, exhibit a highly deterministic response to various types of stimulation. We show that an ensemble of deterministic leaky integrate-and-fire neurons embedded in a spiking noisy environment can attain the correct firing statistics in order to sample from a well-defined target distribution. We provide an analytical derivation of the activation function on the single cell level; for recurrent netwo…
École polytechnique fédérale de Lausanne (EPFL)（洛桑联邦理工学院）
Can we use spiking neural networks (SNN) as generative models of multi-neuronal recordings, while taking into account that most neurons are unobserved? Modeling the unobserved neurons with large pools of hidden spiking neurons leads to severely underconstrained problems that are hard to tackle with maximum likelihood estimation. In this work, we use coarse-graining and mean-field approximations to derive a bottom-up, neuronally-grounded latent variable model (neuLVM), where the activity of the unobserved neurons is reduced to a low-dimensional mesoscopic description. In contrast to previous la…
We present a first-order non-homogeneous Markov model for the interspike-interval density of a continuously stimulated spiking neuron. The model allows the conditional interspike-interval density and the stationary interspike-interval density to be expressed as products of two separate functions, one of which describes only the neuron characteristics, and the other of which describes only the signal characteristics. This allows the use of this model to predict the response when the underlying neuron model is not known or well determined. The approximation shows particularly clearly that signal…
Network of neurons in the brain apply - unlike processors in our current generation of computer hardware - an event-based processing strategy, where short pulses (spikes) are emitted sparsely by neurons to signal the occurrence of an event at a particular point in time. Such spike-based computations promise to be substantially more power-efficient than traditional clocked processing schemes. However it turned out to be surprisingly difficult to design networks of spiking neurons that are able to carry out demanding computations. We present here a new theoretical framework for organizing comput…
Much progress has been made in uncovering the computational capabilities of spiking neural networks. However, spiking neurons will always be more expensive to simulate compared to rate neurons because of the inherent disparity in time scales - the spike duration time is much shorter than the inter-spike time, which is much shorter than any learning time scale. In numerical analysis, this is a classic stiff problem. Spiking neurons are also much more difficult to study analytically. One possible approach to making spiking networks more tractable is to augment mean field activity models with som…
While spike timing has been shown to carry detailed stimulus information at the sensory periphery, its possible role in network computation is less clear. Most models of computation by neural networks are based on population firing rates. In equivalent spiking implementations, firing is assumed to be random such that averaging across populations of neurons recovers the rate-based approach. Recently, however, Denéve and colleagues have suggested that the spiking behavior of neurons may be fundamental to how neuronal networks compute, with precise spike timing determined by each neuron's contrib…
We examine the effects of stochastic input currents on the firing behavior of two excitable neurons coupled with fast excitatory synapses. In such cells (models), typified by the quadratic integrate and fire model, mutual synaptic coupling can cause sustained firing or oscillatory behavior which is necessarily antiphase. Additive Gaussian white noise can transiently terminate the oscillations, hence destroying the stable limit cycle. Further application of the noise may return the system to spiking activity. In a particular noise range, the transition times between the oscillating and the rest…
A major goal of neuroscience, statistical physics and nonlinear dynamics is to understand how brain function arises from the collective dynamics of networks of spiking neurons. This challenge has been chiefly addressed through large-scale numerical simulations. Alternatively, researchers have formulated mean-field theories to gain insight into macroscopic states of large neuronal networks in terms of the collective firing activity of the neurons, or the firing rate. However, these theories have not succeeded in establishing an exact correspondence between the firing rate of the network and the…
We address the problem of learning feedback control where the controller is a network constructed solely of deterministic spiking neurons. In contrast to previous investigations that were based on a spike rate model of the neuron, the control signal here is determined by the precise temporal positions of spikes generated by the output neurons of the network. We model the problem formally as a hybrid dynamical system comprised of a closed loop between a plant and a spiking neuron network. We derive a novel synaptic weight update rule via which the spiking neuron network controller learns to hol…
Networks in the brain consist of different types of neurons. Here we investigate the influence of neuron diversity on the dynamics, phase space structure and computational capabilities of spiking neural networks. We find that already a single neuron of a different type can qualitatively change the network dynamics and that mixed networks may combine the computational capabilities of ones with a single neuron type. We study inhibitory networks of concave leaky (LIF) and convex "anti-leaky" (XIF) integrate-and-fire neurons that generalize irregularly spiking non-chaotic LIF neuron networks. Endo…
A main concern in cognitive neuroscience is to decode the overt neural spike train observations and infer latent representations under neural circuits. However, traditional methods entail strong prior on network structure and hardly meet the demand for real spike data. Here we propose a novel neural network approach called Neuron Activation Network that extracts neural information explicitly from single trial neuron population spike trains. Our proposed method consists of a spatiotemporal learning procedure on sensory environment and a message passing mechanism on population graph, followed by…
This paper describes the Synapto-dendritic Kernel Adapting Neuron (SKAN), a simple spiking neuron model that performs statistical inference and unsupervised learning of spatiotemporal spike patterns. SKAN is the first proposed neuron model to investigate the effects of dynamic synapto-dendritic kernels and demonstrate their computational power even at the single neuron scale. The rule-set defining the neuron is simple there are no complex mathematical operations such as normalization, exponentiation or even multiplication. The functionalities of SKAN emerge from the real-time interaction of si…
Starting from a spectral expansion of the Fokker-Plank equation for the membrane potential density in a network of spiking neurons, a low-dimensional dynamics of the collective firing rate is derived. As a result a $n$-order ordinary differential equation for the network activity can be worked out by taking into account the slowest $n$ modes of the expansion. The resulting low-dimensional dynamics naturally takes into account the strength of the synaptic couplings under the hypothesis of a not too fast changing membrane potential density. By considering only the two slowest modes, the firing r…
Spontaneous fluctuations and stimulus response are essential features of neural functioning but how they are connected is poorly understood. I derive fluctuation-dissipation relations (FDR) between the spontaneous spike and voltage correlations and the firing rate susceptibility for i) the leaky integrate-and-fire (IF) model with white noise; ii) an IF model with arbitrary voltage dependence, an adaptation current, and correlated noise. The FDRs can be used to derive correlation statistics or to infer the system's response from observations of its spontaneous activity.
The relationship between a neuron's complex inputs and its spiking output defines the neuron's coding strategy. This is frequently and effectively modeled phenomenologically by one or more linear filters that extract the components of the stimulus that are relevant for triggering spikes, and a nonlinear function that relates stimulus to firing probability. In many sensory systems, these two components of the coding strategy are found to adapt to changes in the statistics of the inputs, in such a way as to improve information transmission. Here, we show for two simple neuron models how feature …
In machine learning, error back-propagation in multi-layer neural networks (deep learning) has been impressively successful in supervised and reinforcement learning tasks. As a model for learning in the brain, however, deep learning has long been regarded as implausible, since it relies in its basic form on a non-local plasticity rule. To overcome this problem, energy-based models with local contrastive Hebbian learning were proposed and tested on a classification task with networks of rate neurons. We extended this work by implementing and testing such a model with networks of leaky integrate…
This paper models the dynamics of a large set of interacting neurons within the framework of statistical field theory. We use a method initially developed in the context of statistical field theory [44] and later adapted to complex systems in interaction [45][46]. Our model keeps track of individual interacting neurons dynamics but also preserves some of the features and goals of neural field dynamics, such as indexing a large number of neurons by a space variable. Thus, this paper bridges the scale of individual interacting neurons and the macro-scale modelling of neural field theory.
The highly variable dynamics of neocortical circuits observed in vivo have been hypothesized to represent a signature of ongoing stochastic inference but stand in apparent contrast to the deterministic response of neurons measured in vitro. Based on a propagation of the membrane autocorrelation across spike bursts, we provide an analytical derivation of the neural activation function that holds for a large parameter space, including the high-conductance state. On this basis, we show how an ensemble of leaky integrate-and-fire neurons with conductance-based synapses embedded in a spiking enviro…
We revisit the dynamics of a prototypical model of balanced activity in networks of spiking neutrons. A detailed investigation of the thermodynamic limit for fixed density of connections (massive coupling) shows that, when inhibition prevails, the asymptotic regime is not asynchronous but rather characterized by a self-sustained irregular, macroscopic (collective) dynamics. So long as the connectivity is massive, this regime is found in many different setups: leaky as well as quadratic integrate-and-fire neurons; large and small coupling strength; weak and strong external currents.
The brain is composed of complex networks of interacting neurons that express considerable heterogeneity in their physiology and spiking characteristics. How does neural heterogeneity affect macroscopic neural dynamics and how does it contribute to neurodynamic functions? In this letter, we address these questions by studying the macroscopic dynamics of networks of heterogeneous Izhikevich neurons. We derive mean-field equations for these networks and examine how heterogeneity in the spiking thresholds of Izhikevich neurons affects the emergent macroscopic dynamics. Our results suggest that th…
We study an abstracted model of neuronal activity via numerical simulation, and report spatiotemporal pattern formation and critical like dynamics. A population of pulse coupled, discretised, relaxation oscillators is simulated over networks with varying edge density and spatial embedded ness. For intermediate edge density and sufficiently strong spatial embeddedness, we observe a novel spatiotemporal pattern in the field of oscillator phases, visually resembling the surface of a frothing liquid. Increasing the edge density results in critical dynamics, with the distribution of neuronal avalan…
Maintaining the ability to fire sparsely is crucial for information encoding in neural networks. Additionally, spiking homeostasis is vital for spiking neural networks with changing numbers of weights and neurons. We discuss a range of network stabilisation approaches, inspired by homeostatic synaptic plasticity mechanisms reported in the brain. These include weight scaling, and weight change as a function of the network's spiking activity. We tested normalisation of the sum of weights for all neurons, and by neuron type. We examined how this approach affects firing rate and performance on clu…
The effect of a temporally correlated afferent current on the firing rate of a leaky integrate-and-fire (LIF) neuron is studied. This current is characterized in terms of rates, auto and cross-correlations, and correlation time scale $τ_c$ of excitatory and inhibitory inputs. The output rate $ν_{out}$ is calculated in the Fokker-Planck (FP) formalism in the limit of both small and large $τ_c$ compared to the membrane time constant $τ$ of the neuron. By simulations we check the analytical results, provide an interpolation valid for all $τ_c$ and study the neuron's response to rapid changes in t…
We derive rigorous results describing the asymptotic dynamics of a discrete time model of spiking neurons introduced in \cite{BMS}. Using symbolic dynamic techniques we show how the dynamics of membrane potential has a one to one correspondence with sequences of spikes patterns (``raster plots''). Moreover, though the dynamics is generically periodic, it has a weak form of initial conditions sensitivity due to the presence of a sharp threshold in the model definition. As a consequence, the model exhibits a dynamical regime indistinguishable from chaos in numerical experiments.
It is shown that there is no standard spiking neural P system that simulates Turing machines with less than exponential time and space overheads. The spiking neural P systems considered here have a constant number of neurons that is independent of the input length. Following this we construct a universal spiking neural P system with exhaustive use of rules that simulates Turing machines in linear time and has only 10 neurons.
We demonstrate deterministic extensive chaos in the dynamics of large sparse networks of theta neurons in the balanced state. The analysis is based on numerically exact calculations of the full spectrum of Lyapunov exponents, the entropy production rate and the attractor dimension. Extensive chaos is found in inhibitory networks and becomes more intense when an excitatory population is included. We find a strikingly high rate of entropy production that would limit information representation in cortical spike patterns to the immediate stimulus response.
