We consider a threshold-crossing spiking process as a simple model for the activity within a population of neurons. Assuming that these neurons are driven by a common fluctuating input with Gaussian statistics, we evaluate the cross-correlation of spike trains in pairs of model neurons with different thresholds. This correlation function tends to be asymmetric in time, indicating a preference for the neuron with the lower threshold to fire before the one with the higher threshold, even if their inputs are identical. The relationship between these results and spike statistics in other models of neural activity are explored. In particular, we compare our model with an integrate-and-fire model in which the membrane voltage resets following each spike. The qualitative properties of spike cross correlations, emerging from the threshold-crossing model, are similar to those of bursting events in the integrate-and-fire model. This is particularly true for generalized integrate-and-fire models in which spikes tend to occur in bursts as observed, for example, in retinal ganglion cells driven by a rapidly fluctuating visual stimulus. The threshold crossing model thus provides a simple, analytically tractable description of event onsets in these neurons.
The effect of environmental temperature on neuronal spiking behaviors is investigated by numerically simulating the temperature dependence of spiking threshold of the Hodgkin-Huxley neuron subject to synaptic stimulus. We find that the spiking threshold exhibits a global minimum in a "comfortable temperature" range where spike initiation needs weakest synaptic strength, indicating the occurrence of optimal use of synaptic transmission in neural system. We further explore the biophysical origin of this phenomenon in ion channel gating kinetics and also discuss its possible biological relevance …
Some recent papers relate the criticality of complex systems to their maximal capacity of information processing. In the present paper, we consider high dimensional point processes, known as age-dependent Hawkes processes, which have been used to model spiking neural networks. Using mean-field approximation, the response of the network to a stimulus is computed and we provide a notion of stimulus sensitivity. It appears that the maximal sensitivity is achieved in the sub-critical regime, yet almost critical for a range of biologically relevant parameters.
Predictive coding can be regarded as a function which reduces the error between an input signal and a top-down prediction. If reducing the error is equivalent to reducing the influence of stimuli from the environment, predictive coding can be regarded as stimulation avoidance by prediction. Our previous studies showed that action and selection for stimulation avoidance emerge in spiking neural networks through spike-timing dependent plasticity (STDP). In this study, we demonstrate that spiking neural networks with random structure spontaneously learn to predict temporal sequences of stimuli ba…
We investigate the stimulus-dependent tuning properties of a noisy ionic conductance model for intrinsic subthreshold oscillations in membrane potential and associated spike generation. On depolarization by an applied current, the model exhibits subthreshold oscillatory activity with occasional spike generation when oscillations reach the spike threshold. We consider how the amount of applied current, the noise intensity, variation of maximum conductance values and scaling to different temperature ranges alter the responses of the model with respect to voltage traces, interspike intervals and …
Neurons in the nervous system exhibit an outstanding variety of morphological and physiological properties. However, close to threshold, this remarkable richness may be grouped succinctly into two basic types of excitability, often referred to as type I and type II. The dynamical traits of these two neuron types have been extensively characterized. It would be interesting, however, to understand the information-processing consequences of their dynamical properties. To that end, here we determine the differences between the stimulus features inducing firing in type I and in type II neurons. We …
Recurrent neural networks are powerful tools for understanding and modeling computation and representation by populations of neurons. Continuous-variable or "rate" model networks have been analyzed and applied extensively for these purposes. However, neurons fire action potentials, and the discrete nature of spiking is an important feature of neural circuit dynamics. Despite significant advances, training recurrently connected spiking neural networks remains a challenge. We present a procedure for training recurrently connected spiking networks to generate dynamical patterns autonomously, to p…
Humans and other animals behave as if we perform fast Bayesian inference underlying decisions and movement control given uncertain sense data. Here we show that a biophysically realistic model of the subthreshold membrane potential of a single neuron can exactly compute the numerator in Bayes rule for inferring the Poisson parameter of a sensory spike train. A simple network of spiking neurons can construct and represent the Bayesian posterior density of a parameter of an external cause that affects the Poisson parameter, accurately and in real time.
When two stimuli are present in the receptive field of a V4 neuron, the firing rate response is between the weakest and strongest response elicited by each of the stimuli alone (Reynolds et al, 1999, Journal of Neuroscience 19:1736-1753). When attention is directed towards the stimulus eliciting the strongest response (the preferred stimulus), the response to the pair is increased, whereas the response decreases when attention is directed to the other stimulus (the poor stimulus). These experimental results were reproduced in a model of a V4 neuron under the assumption that attention modulates…
The response of a neuron to synaptic input strongly depends on whether or not it has just emitted a spike. We propose a neuron model that after spike emission exhibits a partial response to residual input charges and study its collective network dynamics analytically. We uncover a novel desynchronization mechanism that causes a sequential desynchronization transition: In globally coupled neurons an increase in the strength of the partial response induces a sequence of bifurcations from states with large clusters of synchronously firing neurons, through states with smaller clusters to completel…
We consider a wide class of spiking neuron models, defined by rather general set of conditions typical for basic models like leaky integrate and fire, or binding neuron model. A neuron is fed with a point renewal process. A relation between the three probability density functions (pdf): (i) pdf of input interspike intervals, (ii) pdf of output interspike intervals of a neuron with instantaneous feedback and (iii) pdf for that same neuron without feedback is derived. This allows to calculate any of the three pdfs provided the another two are given. Similar relation between corresponding means a…
We study the effect of intrinsic heterogeneity on the activity of a population of leaky integrate-and-fire neurons. By rescaling the dynamical equation, we derive mathematical relations between multiple neuronal parameters and a fluctuating input noise. To this end, common input to heterogeneous neurons is conceived as an identical noise with neuron-specific mean and variance. As a consequence, the neuronal output rates can differ considerably, and their relative spike timing becomes desynchronized. This theory can quantitatively explain some recent experimental findings.
The influence of temperature on neuronal excitability is studied by numerical simulations on the spiking threshold characteristics of bushy cells in cochlear nucleus periodically stimulated by synaptic currents. The results reveal that there is a cut-off frequency for the spiking of bushy cell in a specific temperature environment, corresponding to the existence of a critical temperature for the neuron to respond with real spikes to the synaptic stimulus of a given frequency, due to the finiteness of spike width. An optimal temperature range for neuronal spiking is also found for a specific st…
Highly connected recurrent neural networks often produce chaotic dynamics, meaning their precise activity is sensitive to small perturbations. What are the consequences for how such networks encode streams of temporal stimuli? On the one hand, chaos is a strong source of randomness, suggesting that small changes in stimuli will be obscured by intrinsically generated variability. On the other hand, recent work shows that the type of chaos that occurs in spiking networks can have a surprisingly low-dimensional structure, suggesting that there may be "room" for fine stimulus features to be precis…
Finite-sized populations of spiking elements are fundamental to brain function, but also used in many areas of physics. Here we present a theory of the dynamics of finite-sized populations of spiking units, based on a quasi-renewal description of neurons with adaptation. We derive an integral equation with colored noise that governs the stochastic dynamics of the population activity in response to time-dependent stimulation and calculate the spectral density in the asynchronous state. We show that systems of coupled populations with adaptation can generate a frequency band in which sensory inf…
Neurons in the primate visual cortices show a wide range of stimulus selectivity. Some neurons respond to only a small fraction of stimulus images, whereas others respond to many stimulus images in a non-selective manner. It is unclear how stimulus selective and non-selective neurons contribute to visual object recognition. Herein, we examined the relationship between stimulus selectivity and the effect of deletion of units on task performance using fully a connected layer of two types of deep convolutional neural networks (DCNNs). Deleting a stimulus selective unit caused slight improvements …
Networks of model neurons with balanced recurrent excitation and inhibition produce irregular and asynchronous spiking activity. We extend the analysis of balanced networks to include the known dependence of connection probability on the spatial separation between neurons. In the continuum limit we derive that stable, balanced firing rate solutions require that the spatial spread of external inputs be broader than that of recurrent excitation, which in turn must be broader than or equal to that of recurrent inhibition. For finite size networks we investigate the pattern forming dynamics arisin…
A long tradition in theoretical neuroscience casts sensory processing in the brain as the process of inferring the maximally consistent interpretations of imperfect sensory input. Recently it has been shown that Gamma-band inhibition can enable neural attractor networks to approximately carry out such a sampling mechanism. In this paper we propose a novel neural network model based on irregular gating inhibition, show analytically how it implements a Monte-Carlo Markov Chain (MCMC) sampler, and describe how it can be used to model networks of both neural attractors as well as of single spiking…
Many central neurons, and in particular certain brainstem aminergic neurons exhibit spontaneous and fairly regular spiking with frequencies of order a few Hz. A large number of ion channel types contribute to such spiking so that accurate modeling of spike generation leads to the requirement of solving very large systems of differential equations, ordinary in the first instance. Since analysis of spiking behavior when many synaptic inputs are active adds further to the number of components, it is useful to have simplified mathematical models of spiking in such neurons so that, for example, sto…
At functional scales, cortical behavior results from the complex interplay of a large number of excitable cells operating in noisy environments. Such systems resist to mathematical analysis, and computational neurosciences have largely relied on heuristic partial (and partially justified) macroscopic models, which successfully reproduced a number of relevant phenomena. The relationship between these macroscopic models and the spiking noisy dynamics of the underlying cells has since then been a great endeavor. Based on recent mean-field reductions for such spiking neurons, we present here {a pr…
We examine whether a single biophysical cortical circuit model can explain both spiking and perceptual variability. We consider perceptual rivalry, which provides a window into intrinsic neural processing since neural activity in some brain areas is correlated to the alternating perception rather than the constant ambiguous stimulus. The prevalent theory for spiking variability is a chaotic attractor called the balanced state; whereas, the source of perceptual variability is an open question. We present a dynamical model with a chaotic attractor that explains both spiking and perceptual variab…
The coding properties of cells with different types of receptive fields have been studied for decades. ON-type neurons fire in response to positive fluctuations of the time-dependent stimulus, whereas OFF cells are driven by negative stimulus segments. Biphasic cells, in turn, are selective to up/down or down/up stimulus upstrokes. In this paper, we explore the way in which different receptive fields affect the firing statistics of Poisson neuron models, when driven with slow stimuli. We find analytical expressions for the time-dependent peri-stimulus time histogram and the inter-spike interva…
Limitations in processing capabilities and memory of today's computers make spiking neuron-based (human) whole-brain simulations inevitably characterized by a compromise between bio-plausibility and computational cost. It translates into brain models composed of a reduced number of neurons and a simplified neuron's mathematical model, leading to the search for new simulation strategies. Taking advantage of the sparse character of brain-like computation, the event-driven technique could represent a way to carry out efficient simulation of large-scale Spiking Neural Networks (SNN). The recent Le…
Neurons in cortical circuits exhibit coordinated spiking activity, and can produce correlated synchronous spikes during behavior and cognition. We recently developed a method for estimating the dynamics of correlated ensemble activity by combining a model of simultaneous neuronal interactions (e.g., a spin-glass model) with a state-space method (Shimazaki et al. 2012 PLoS Comput Biol 8 e1002385). This method allows us to estimate stimulus-evoked dynamics of neuronal interactions which is reproducible in repeated trials under identical experimental conditions. However, the method may not be sui…
Several studies with brain signals suggested that bottom-up and top-down influences are exerted through distinct frequency bands among visual cortical areas. It has been recently shown that theta and gamma rhythms subserve feedforward, whereas the feedback influence is dominated by the alpha-beta rhythm in primates. A few theoretical models for reproducing these effects have been proposed so far. Here we show that a simple but biophysically plausible two-network motif composed of spiking-neuron models and chemical synapses can exhibit feedforward and feedback influences through distinct freque…
We study the reconstruction of visual stimuli from spike trains, recording simultaneously from the two H1 neurons located in the lobula plate of the fly Chrysomya megacephala. The fly views two types of stimuli, corresponding to rotational and translational displacements. If the reconstructed stimulus is to be represented by a Volterra series and correlations between spikes are to be taken into account, first order expansions are insufficient and we have to go to second order, at least. In this case higher order correlation functions have to be manipulated, whose size may become prohibitively …
Top-down feedback in cortex is critical for guiding sensory processing, which has prominently been formalized in the theory of hierarchical predictive coding (hPC). However, experimental evidence for error units, which are central to the theory, is inconclusive, and it remains unclear how hPC can be implemented with spiking neurons. To address this, we connect hPC to existing work on efficient coding in balanced networks with lateral inhibition, and predictive computation at apical dendrites. Together, this work points to an efficient implementation of hPC with spiking neurons, where predictio…
We present a theoretical study aiming at model fitting for sensory neurons. Conventional neural network training approaches are not applicable to this problem due to lack of continuous data. Although the stimulus can be considered as a smooth time dependent variable, the associated response will be a set of neural spike timings (roughly the instants of successive action potential peaks) which have no amplitude information. A recurrent neural network model can be fitted to such a stimulus-response data pair by using maximum likelihood estimation method where the likelihood function is derived f…
To compensate for sensory processing delays, the visual system must make predictions to ensure timely and appropriate behaviors. Recent work has found predictive information about the stimulus in neural populations early in vision processing, starting in the retina. However, to utilize this information, cells downstream must in turn be able to read out the predictive information from the spiking activity of retinal ganglion cells. Here we investigate whether a downstream cell could learn efficient encoding of predictive information in its inputs in the absence of other instructive signals, fro…
Neural populations exposed to a certain stimulus learn to represent it better. However, the process that leads local, self-organized rules to do so is unclear. We address the question of how can a neural periodic input be learned and use the Differential Hebbian Learning framework, coupled with a homeostatic mechanism to derive two self-consistency equations that lead to increased responses to the same stimulus. Although all our simulations are done with simple Leaky-Integrate and Fire neurons and standard Spiking Time Dependent Plasticity learning rules, our results can be easily interpreted …
Correlations in sensory neural networks have both extrinsic and intrinsic origins. Extrinsic or stimulus correlations arise from shared inputs to the network, and thus depend strongly on the stimulus ensemble. Intrinsic or noise correlations reflect biophysical mechanisms of interactions between neurons, which are expected to be robust to changes of the stimulus ensemble. Despite the importance of this distinction for understanding how sensory networks encode information collectively, no method exists to reliably separate intrinsic interactions from extrinsic correlations in neural activity da…
In this article, we study optimal control problems of spiking neurons whose dynamics are described by a phase model. We design minimum-power current stimuli (controls) that lead to targeted spiking times of neurons, where the cases with unbounded and bounded control amplitude are considered. We show that theoretically the spiking period of a neuron, modeled by phase dynamics, can be arbitrarily altered by a smooth control. However, if the control amplitude is bounded, the range of possible spiking times is constrained and determined by the bound, and feasible spiking times are optimally achiev…
We propose a model of an adaptive network of spiking neurons that gives rise to a hypernetwork of its dynamic states at the upper level of description. Left to itself, the network exhibits a sequence of transient clustering which relates to a traffic in the hypernetwork in the form of a random walk. Receiving inputs the system is able to generate reproducible sequences corresponding to stimulus-specific paths in the hypernetwork. We illustrate these basic notions by a simple network of discrete-time spiking neurons together with its FPGA realization and analyze their properties.
In this study, we analyzed the activity of monkey V1 neurons responding to grating stimuli of different orientations using inference methods for a time-dependent Ising model. The method provides optimal estimation of time-dependent neural interactions with credible intervals according to the sequential Bayes estimation algorithm. Furthermore, it allows us to trace dynamics of macroscopic network properties such as entropy, sparseness, and fluctuation. Here we report that, in all examined stimulus conditions, pairwise interactions contribute to increasing sparseness and fluctuation. We then dem…
Neural coding is a field of study that concerns how sensory information is represented in the brain by networks of neurons. The link between external stimulus and neural response can be studied from two parallel points of view. The first, neural encoding refers to the mapping from stimulus to response, and primarily focuses on understanding how neurons respond to a wide variety of stimuli, and on constructing models that accurately describe the stimulus-response relationship. Neural decoding, on the other hand, refers to the reverse mapping, from response to stimulus, where the challenge is to…
Spiking neural network is a type of artificial neural network in which neurons communicate between each other with spikes. Spikes are identical Boolean events characterized by the time of their arrival. A spiking neuron has internal dynamics and responds to the history of inputs as opposed to the current inputs only. Because of such properties a spiking neural network has rich intrinsic capabilities to process spatiotemporal data. However, because the spikes are discontinuous 'yes or no' events, it is not trivial to apply traditional training procedures such as gradient descend to the spiking …
We demonstrate a spiking neural network for navigation motivated by the chemotaxis network of Caenorhabditis elegans. Our network uses information regarding temporal gradients in the tracking variable's concentration to make navigational decisions. The gradient information is determined by mimicking the underlying mechanisms of the ASE neurons of C. elegans. Simulations show that our model is able to forage and track a target set-point in extremely noisy environments. We develop a VLSI implementation for the main gradient detector neurons, which could be integrated with standard comparator cir…
We establish a general linear response relation for spiking neuronal networks, based on chains with unbounded memory. This relation allows us to predict the influence of a weak amplitude time-dependent external stimuli on spatio-temporal spike correlations, from the spontaneous statistics (without stimulus) in a general context where the memory in spike dynamics can extend arbitrarily far in the past. Using this approach, we show how linear response is explicitly related to neuronal dynamics with an example, the gIF model, introduced by M. Rudolph and A. Destexhe. This example illustrates the …
We show that dynamical gain modulation of neurons' stimulus response is described as an information-theoretic cycle that generates entropy associated with the stimulus-related activity from entropy produced by the modulation. To articulate this theory, we describe stimulus-evoked activity of a neural population based on the maximum entropy principle with constraints on two types of overlapping activities, one that is controlled by stimulus conditions and the other, termed internal activity, that is regulated internally in an organism. We demonstrate that modulation of the internal activity rea…
Spike correlations between neurons are ubiquitous in the cortex, but their role is at present not understood. Here we describe the firing response of a leaky integrate-and-fire neuron (LIF) when it receives a temporarily correlated input generated by presynaptic correlated neuronal populations. Input correlations are characterized in terms of the firing rates, Fano factors, correlation coefficients and correlation timescale of the neurons driving the target neuron. We show that the sum of the presynaptic spike trains cannot be well described by a Poisson process. Solutions of the output firing…
We present a first-order non-homogeneous Markov model for the interspike-interval density of a continuously stimulated spiking neuron. The model allows the conditional interspike-interval density and the stationary interspike-interval density to be expressed as products of two separate functions, one of which describes only the neuron characteristics, and the other of which describes only the signal characteristics. This allows the use of this model to predict the response when the underlying neuron model is not known or well determined. The approximation shows particularly clearly that signal…
Network of neurons in the brain apply - unlike processors in our current generation of computer hardware - an event-based processing strategy, where short pulses (spikes) are emitted sparsely by neurons to signal the occurrence of an event at a particular point in time. Such spike-based computations promise to be substantially more power-efficient than traditional clocked processing schemes. However it turned out to be surprisingly difficult to design networks of spiking neurons that are able to carry out demanding computations. We present here a new theoretical framework for organizing comput…
Most neurons in peripheral sensory pathways initially respond vigorously when a preferred stimulus is presented, but adapt as stimulation continues. It is unclear how this phenomenon affects stimulus representation in the later stages of cortical sensory processing. Here, we show that a temporally sparse and reliable stimulus representation develops naturally in a network with adapting neurons. We find that cellular adaptation plays a critical role in the transient reduction of the trial-by-trial variability of cortical spiking, providing an explanation for a wide-spread and hitherto unexplain…
We address the problem of learning feedback control where the controller is a network constructed solely of deterministic spiking neurons. In contrast to previous investigations that were based on a spike rate model of the neuron, the control signal here is determined by the precise temporal positions of spikes generated by the output neurons of the network. We model the problem formally as a hybrid dynamical system comprised of a closed loop between a plant and a spiking neuron network. We derive a novel synaptic weight update rule via which the spiking neuron network controller learns to hol…
In this article, our wish is to demystify some aspects of coding with spike-timing, through a simple review of well-understood technical facts regarding spike coding. The goal is to help better understanding to which extend computing and modelling with spiking neuron networks can be biologically plausible and computationally efficient. We intentionally restrict ourselves to a deterministic dynamics, in this review, and we consider that the dynamics of the network is defined by a non-stochastic mapping. This allows us to stay in a rather simple framework and to propose a review with concrete nu…
We are interested in characterization of synchronization transitions of bursting neurons in the frequency domain. Instantaneous population firing rate (IPFR) $R(t)$, which is directly obtained from the raster plot of neural spikes, is often used as a realistic collective quantity describing population activities in both the computational and the experimental neuroscience. For the case of spiking neurons, a realistic time-domain order parameter, based on $R(t)$, was introduced in our recent work to characterize the spike synchronization transition. Unlike the case of spiking neurons, the IPFR $…
The dynamical responses of complex neuronal networks to external stimulus injected on a \emph{single} neuron are investigated. Stimulating the largest-degree neuron in the network, it is found that as the intensity of the stimulus increases, the network will be transiting from the resting to firing states and then restoring to the resting state, showing a bounded firing region in the parameter space. Furthermore, it is found that as the coupling strength decreases, the firing region is gradually expanded and, at the weak couplings, separated into disconnected subregions. By a simplified networ…
This paper describes the Synapto-dendritic Kernel Adapting Neuron (SKAN), a simple spiking neuron model that performs statistical inference and unsupervised learning of spatiotemporal spike patterns. SKAN is the first proposed neuron model to investigate the effects of dynamic synapto-dendritic kernels and demonstrate their computational power even at the single neuron scale. The rule-set defining the neuron is simple there are no complex mathematical operations such as normalization, exponentiation or even multiplication. The functionalities of SKAN emerge from the real-time interaction of si…
Starting from a spectral expansion of the Fokker-Plank equation for the membrane potential density in a network of spiking neurons, a low-dimensional dynamics of the collective firing rate is derived. As a result a $n$-order ordinary differential equation for the network activity can be worked out by taking into account the slowest $n$ modes of the expansion. The resulting low-dimensional dynamics naturally takes into account the strength of the synaptic couplings under the hypothesis of a not too fast changing membrane potential density. By considering only the two slowest modes, the firing r…
Spontaneous fluctuations and stimulus response are essential features of neural functioning but how they are connected is poorly understood. I derive fluctuation-dissipation relations (FDR) between the spontaneous spike and voltage correlations and the firing rate susceptibility for i) the leaky integrate-and-fire (IF) model with white noise; ii) an IF model with arbitrary voltage dependence, an adaptation current, and correlated noise. The FDRs can be used to derive correlation statistics or to infer the system's response from observations of its spontaneous activity.
The relationship between a neuron's complex inputs and its spiking output defines the neuron's coding strategy. This is frequently and effectively modeled phenomenologically by one or more linear filters that extract the components of the stimulus that are relevant for triggering spikes, and a nonlinear function that relates stimulus to firing probability. In many sensory systems, these two components of the coding strategy are found to adapt to changes in the statistics of the inputs, in such a way as to improve information transmission. Here, we show for two simple neuron models how feature …
