The macroscopic dynamics of large populations of neurons can be mathematically analyzed using low-dimensional firing-rate or neural-mass models. However, these models fail to capture spike synchronization effects of stochastic spiking neurons such as the non-stationary population response to rapidly changing stimuli. Here, we derive low-dimensional firing-rate models for homogeneous populations of general renewal-type neurons, including integrate-and-fire models driven by white noise. Renewal models account for neuronal refractoriness and spike synchronization dynamics. The derivation is based on an eigenmode expansion of the associated refractory density equation, which generalizes previous spectral methods for Fokker-Planck equations to arbitrary renewal models. We find a simple relation between the eigenvalues, which determine the characteristic time scales of the firing rate dynamics, and the Laplace transform of the interspike interval density or the survival function of the renewal process. Analytical expressions for the Laplace transforms are readily available for many renewal models including the leaky integrate-and-fire model. Retaining only the first eigenmode yields already an adequate low-dimensional approximation of the firing-rate dynamics that captures spike synchronization effects and fast transient dynamics at stimulus onset. We explicitly demonstrate the validity of our model for a large homogeneous population of Poisson neurons with absolute refractoriness, and other renewal models that admit an explicit analytical calculation of the eigenvalues. The here presented eigenmode expansion provides a systematic framework for novel firing-rate models in computational neuroscience based on spiking neuron dynamics with refractoriness.
Fast-spiking (FS) interneurons in the brain are self-innervated by powerful inhibitory GABAergic autaptic connections. By computational modelling, we investigate how autaptic inhibition regulates the firing response of such interneurons. Our results indicate that autaptic inhibition both boosts the current threshold for action potential generation as well as modulates the input-output gain of FS interneurons. The autaptic transmission delay is identified as a key parameter that controls the firing patterns and determines multistability regions of FS interneurons. Furthermore, we observe that n…
Recurrent neural networks are powerful tools for understanding and modeling computation and representation by populations of neurons. Continuous-variable or "rate" model networks have been analyzed and applied extensively for these purposes. However, neurons fire action potentials, and the discrete nature of spiking is an important feature of neural circuit dynamics. Despite significant advances, training recurrently connected spiking neural networks remains a challenge. We present a procedure for training recurrently connected spiking networks to generate dynamical patterns autonomously, to p…
Heterogeneity of neural attributes has recently gained a lot of attention and is increasing recognized as a crucial feature in neural processing. Despite its importance, this physiological feature has traditionally been neglected in theoretical studies of cortical neural networks. Thus, there is still a lot unknown about the consequences of cellular and circuit heterogeneity in spiking neural networks. In particular, combining network or synaptic heterogeneity and intrinsic heterogeneity has yet to be considered systematically despite the fact that both are known to exist and likely have signi…
We consider a wide class of spiking neuron models, defined by rather general set of conditions typical for basic models like leaky integrate and fire, or binding neuron model. A neuron is fed with a point renewal process. A relation between the three probability density functions (pdf): (i) pdf of input interspike intervals, (ii) pdf of output interspike intervals of a neuron with instantaneous feedback and (iii) pdf for that same neuron without feedback is derived. This allows to calculate any of the three pdfs provided the another two are given. Similar relation between corresponding means a…
Strong inhibitory input to neurons, which occurs in balanced states of neural networks, increases synaptic current fluctuations. This has led to the assumption that inhibition contributes to the high spike-firing irregularity observed in vivo. We used single compartment neuronal models with time-correlated (due to synaptic filtering) and state-dependent (due to reversal potentials) input to demonstrate that inhibitory input acts to decrease membrane potential fluctuations, a result that cannot be achieved with simplified neural input models. To clarify the effects on spike-firing regularity, w…
How do neuronal populations encode time-dependent stimuli in their population firing rate? To address this question, I consider the quasi-renewal equation and the event-based expansion, two theoretical approximations proposed recently, and test these against peri-stimulus time histograms from L2-3 pyramidal cells in vitro. Parameters are optimized by gradient descent to best match the firing rate output given the current input. The fitting method can estimate single-neuron parameters that are normally obtained either with intracellular recordings or with individual spike trains. I find that qu…
A central question in neuroscience is to understand how noisy firing patterns are used to transmit information. Because neural spiking is noisy, spiking patterns are often quantified via pairwise correlations, or the probability that two cells will spike coincidentally, above and beyond their baseline firing rate. One observation frequently made in experiments, is that correlations can increase systematically with firing rate. Theoretical studies have determined that stimulus-dependent correlations that increase with firing rate can have beneficial effects on information coding; however, we st…
The dominant modeling framework for understanding cortical computations are heuristic firing rate models. Despite their success, these models fall short to capture spike synchronization effects, to link to biophysical parameters and to describe finite-size fluctuations. In this opinion article, we propose that the refractory density method (RDM), also known as age-structured population dynamics or quasi-renewal theory, yields a powerful theoretical framework to build rate-based models for mesoscopic neural populations from realistic neuron dynamics at the microscopic level. We review recent ad…
We examine the effects of stochastic input currents on the firing behavior of two excitable neurons coupled with fast excitatory synapses. In such cells (models), typified by the quadratic integrate and fire model, mutual synaptic coupling can cause sustained firing or oscillatory behavior which is necessarily antiphase. Additive Gaussian white noise can transiently terminate the oscillations, hence destroying the stable limit cycle. Further application of the noise may return the system to spiking activity. In a particular noise range, the transition times between the oscillating and the rest…
Maintaining the ability to fire sparsely is crucial for information encoding in neural networks. Additionally, spiking homeostasis is vital for spiking neural networks with changing numbers of weights and neurons. We discuss a range of network stabilisation approaches, inspired by homeostatic synaptic plasticity mechanisms reported in the brain. These include weight scaling, and weight change as a function of the network's spiking activity. We tested normalisation of the sum of weights for all neurons, and by neuron type. We examined how this approach affects firing rate and performance on clu…
Starting from a spectral expansion of the Fokker-Plank equation for the membrane potential density in a network of spiking neurons, a low-dimensional dynamics of the collective firing rate is derived. As a result a $n$-order ordinary differential equation for the network activity can be worked out by taking into account the slowest $n$ modes of the expansion. The resulting low-dimensional dynamics naturally takes into account the strength of the synaptic couplings under the hypothesis of a not too fast changing membrane potential density. By considering only the two slowest modes, the firing r…
In this work we consider a periodically forced generic integrate-and-fire model with a unique attracting equilibrium in the subthreshold dynamics and study the dependence of the firing-rate on the frequency of the drive. In an earlier study we have obtained rigorous results on the bifurcation structure in such systems, with emphasis on the relation between the firing-rate and the rotation number of the existing periodic orbits. In this work we study how these bifurcation structures behave upon variation of the frequency of the input. This allows us to show that the dependence of the firing-rat…
Neuronal voltage dynamics of regularly firing neurons typically has one stable attractor: either a fixed point (like in the subthreshold regime) or a limit cycle that defines the tonic firing of action potentials (in the suprathreshold regime). In two of the three spike onset bifurcation sequences that are known to give rise to all-or-none type action potentials, however, the resting-state fixpoint and limit cycle spiking can coexist in an intermediate regime, resulting in bistable dynamics. Here, noise can induce switches between the attractors, i.e., between rest and spiking, and thus increa…
We present a simple Markov model of spiking neural dynamics that can be analytically solved to characterize the stochastic dynamics of a finite-size spiking neural network. We give closed-form estimates for the equilibrium distribution, mean rate, variance and autocorrelation function of the network activity. The model is applicable to any network where the probability of firing of a neuron in the network only depends on the number of neurons that fired in a previous temporal epoch. Networks with statistically homogeneous connectivity and membrane and synaptic time constants that are not exces…
Spiking neural networks (SNNs) has attracted much attention due to its great potential of modeling time-dependent signals. The firing rate of spiking neurons is decided by control rate which is fixed manually in advance, and thus, whether the firing rate is adequate for modeling actual time series relies on fortune. Though it is demanded to have an adaptive control rate, it is a non-trivial task because the control rate and the connection weights learned during the training process are usually entangled. In this paper, we show that the firing rate is related to the eigenvalue of the spike gene…
A common approach to interpreting spiking activity is based on identifying the firing fields---regions in physical or configuration spaces that elicit responses of neurons. Common examples include hippocampal place cells that fire at preferred locations in the navigated environment, head direction cells that fire at preferred orientations of the animal's head, view cells that respond to preferred spots in the visual field, etc. In all these cases, firing fields were discovered empirically, by trial and error. We argue that the existence and a number of properties of the firing fields can be es…
We consider a class of spiking neuronal models, defined by a set of conditions typical for basic threshold-type models, such as the leaky integrate-and-fire or the binding neuron model and also for some artificial neurons. A neuron is fed with a Poisson process. Each output impulse is applied to the neuron itself after a finite delay $Δ$. This impulse acts as being delivered through a fast Cl-type inhibitory synapse. We derive a general relation which allows calculating exactly the probability density function (pdf) $p(t)$ of output interspike intervals of a neuron with feedback based on known…
Heterogeneity of firing rate statistics is known to have severe consequences on neural coding. Recent experimental recordings in weakly electric fish indicate that the distribution-width of superficial pyramidal cell firing rates (trial- and time-averaged) in the electrosensory lateral line lobe (ELL) depends on the stimulus, and also that network inputs can mediate changes in the firing rate distribution across the population. We previously developed theoretical methods to understand how two attributes (synaptic and intrinsic heterogeneity) interact and alter the firing rate distribution in a…
Chemical and electrical synapses shape the dynamics of neuronal networks. Numerous theoretical studies have investigated how each of these types of synapses contributes to the generation of neuronal oscillations, but their combined effect is less understood. This limitation is further magnified by the impossibility of traditional neuronal mean-field models ---also known as firing rate models, or firing rate equations--- to account for electrical synapses. Here we introduce a novel firing rate model that exactly describes the mean field dynamics of heterogeneous populations of quadratic integra…
Spike correlations between neurons are ubiquitous in the cortex, but their role is at present not understood. Here we describe the firing response of a leaky integrate-and-fire neuron (LIF) when it receives a temporarily correlated input generated by presynaptic correlated neuronal populations. Input correlations are characterized in terms of the firing rates, Fano factors, correlation coefficients and correlation timescale of the neurons driving the target neuron. We show that the sum of the presynaptic spike trains cannot be well described by a Poisson process. Solutions of the output firing…
Mounting evidence in recent years suggests that astrocytes, a sub-type of glial cells, not only serve metabolic and structural support for neurons and synapses but also play critical roles in regulation of proper functioning of the nervous system. In this work, we investigate the effect of astrocyte on the spontaneous firing activity of a neuron through a combined model which includes a neuron-astrocyte pair. First, we show that an astrocyte may provide a kind of multistability in neuron dynamics by inducing different firing modes such as random and bursty spiking. Then, we identify the underl…
The state space of a conventional Hopfield network typically exhibits many different attractors of which only a small subset satisfy constraints between neurons in a globally optimal fashion. It has recently been demonstrated that combining Hebbian learning with occasional alterations of normal neural states avoids this problem by means of self-organized enlargement of the best basins of attraction. However, so far it is not clear to what extent this process of self-optimization is also operative in real brains. Here we demonstrate that it can be transferred to more biologically plausible neur…
Spiking neural networks (SNNs) communicate through the all-or-none spiking activity of neurons. However, fitting the large number of SNN model parameters to observed neural activity patterns, for example, in biological experiments, remains a challenge. Previous work using genetic algorithm (GA) optimisation on a specific efficient SNN model, using the Izhikevich neuronal model, was limited to a single parameter and objective. This work applied a version of GA, called non-dominated sorting GA (NSGA-III), to demonstrate the feasibility of performing multi-objective optimisation on the same SNN, …
A major goal of neuroscience, statistical physics and nonlinear dynamics is to understand how brain function arises from the collective dynamics of networks of spiking neurons. This challenge has been chiefly addressed through large-scale numerical simulations. Alternatively, researchers have formulated mean-field theories to gain insight into macroscopic states of large neuronal networks in terms of the collective firing activity of the neurons, or the firing rate. However, these theories have not succeeded in establishing an exact correspondence between the firing rate of the network and the…
A synfire chain is a simple neural network model which can propagate stable synchronous spikes called a pulse packet and widely researched. However how synfire chains coexist in one network remains to be elucidated. We have studied the activity of a layered associative network of Leaky Integrate-and-Fire neurons in which connection we embed memory patterns by the Hebbian Learning. We analyzed their activity by the Fokker-Planck method. In our previous report, when a half of neurons belongs to each memory pattern (memory pattern rate $F=0.5$), the temporal profiles of the network activity is sp…
Mean field theory is a device to analyze the collective behavior of a dynamical system comprising many interacting particles. The theory allows to reduce the behavior of the system to the properties of a handful of parameters. In neural circuits, these parameters are typically the firing rates of distinct, homogeneous subgroups of neurons. Knowledge of the firing rates under conditions of interest can reveal essential information on both the dynamics of neural circuits and the way they can subserve brain function. The goal of this chapter is to provide an elementary introduction to the mean fi…
Networks of model neurons with balanced recurrent excitation and inhibition produce irregular and asynchronous spiking activity. We extend the analysis of balanced networks to include the known dependence of connection probability on the spatial separation between neurons. In the continuum limit we derive that stable, balanced firing rate solutions require that the spatial spread of external inputs be broader than that of recurrent excitation, which in turn must be broader than or equal to that of recurrent inhibition. For finite size networks we investigate the pattern forming dynamics arisin…
The background activity of a cortical neural network is modeled by a homogeneous integrate-and-fire network with unreliable inhibitory synapses. Numerical and analytical calculations show that the network relaxes into a stationary state of high attention. The majority of the neurons has a membrane potential just below the threshold; as a consequence the network can react immediately - on the time scale of synaptic transmission- on external pulses. The neurons fire with a low rate and with a broad distribution of interspike intervals. Firing events of the total network are correlated over short…
The response of a neuron to synaptic input strongly depends on whether or not it has just emitted a spike. We propose a neuron model that after spike emission exhibits a partial response to residual input charges and study its collective network dynamics analytically. We uncover a novel desynchronization mechanism that causes a sequential desynchronization transition: In globally coupled neurons an increase in the strength of the partial response induces a sequence of bifurcations from states with large clusters of synchronously firing neurons, through states with smaller clusters to completel…
Finite-sized populations of spiking elements are fundamental to brain function, but also used in many areas of physics. Here we present a theory of the dynamics of finite-sized populations of spiking units, based on a quasi-renewal description of neurons with adaptation. We derive an integral equation with colored noise that governs the stochastic dynamics of the population activity in response to time-dependent stimulation and calculate the spectral density in the asynchronous state. We show that systems of coupled populations with adaptation can generate a frequency band in which sensory inf…
While spike timing has been shown to carry detailed stimulus information at the sensory periphery, its possible role in network computation is less clear. Most models of computation by neural networks are based on population firing rates. In equivalent spiking implementations, firing is assumed to be random such that averaging across populations of neurons recovers the rate-based approach. Recently, however, Denéve and colleagues have suggested that the spiking behavior of neurons may be fundamental to how neuronal networks compute, with precise spike timing determined by each neuron's contrib…
Recurrently coupled networks of inhibitory neurons robustly generate oscillations in the gamma band. Nonetheless, the corresponding Wilson-Cowan type firing rate equation for such an inhibitory population does not generate such oscillations without an explicit time delay. We show that this discrepancy is due to a voltage-dependent spike-synchronization mechanism inherent in networks of spiking neurons which is not captured by standard firing rate equations. Here we investigate an exact low-dimensional description for a network of heterogeneous canonical type-I inhibitory neurons which includes…
The impairment of cognitive function in Alzheimer's is clearly correlated to synapse loss. However, the mechanisms underlying this correlation are only poorly understood. Here, we investigate how the loss of excitatory synapses in sparsely connected random networks of spiking excitatory and inhibitory neurons alters their dynamical characteristics. Beyond the effects on the network's activity statistics, we find that the loss of excitatory synapses on excitatory neurons shifts the network dynamic towards the stable regime. The decrease in sensitivity to small perturbations to time varying inpu…
Primary vestibular neurons are categorized as either regularly or irregularly firing afferents that use rate and temporal sensory encoding strategies, respectively. While many factors influence firing in these neurons, recent work in mammalian vestibular afferents has demonstrated a rich diversity in ion channels that drive spiking regularity. Here, I review key ionic currents studied in vitro and demonstrate how they may enable sensory encoding strategies demonstrated in vivo.
Phase transitions and critical behavior are crucial issues both in theoretical and experimental neuroscience. We report analytic and computational results about phase transitions and self-organized criticality (SOC) in networks with general stochastic neurons. The stochastic neuron has a firing probability given by a smooth monotonic function $Φ(V)$ of the membrane potential $V$, rather than a sharp firing threshold. We find that such networks can operate in several dynamic regimes (phases) depending on the average synaptic weight and the shape of the firing function $Φ$. In particular, we enc…
Neurons process information by transforming barrages of synaptic inputs into spiking activity. Synaptic inhibition suppresses the output firing activity of a neuron, and is commonly classified as having a subtractive or divisive effect on a neuron's output firing activity. Subtractive inhibition can narrow the range of inputs that evoke spiking activity by eliminating responses to non-preferred inputs. Divisive inhibition is a form of gain control: it modifies firing rates while preserving the range of inputs that evoke firing activity. Since these two "modes" of inhibition have distinct impac…
Self-sustained activity in the brain is observed in the absence of external stimuli and contributes to signal propagation, neural coding, and dynamic stability. It also plays an important role in cognitive processes. In this work, by means of studying intracellular recordings from CA1 neurons in rats and results from numerical simulations, we demonstrate that self-sustained activity presents high variability of patterns, such as low neural firing rates and activity in the form of small-bursts in distinct neurons. In our numerical simulations, we consider random networks composed of coupled, ad…
Two-photon imaging of calcium indicators allows simultaneous recording of responses of hundreds of neurons over hours and even days, but provides a relatively indirect measure of their spiking activity. Existing deconvolution algorithms attempt to recover spikes from observed imaging data, which are then commonly subjected to the same analyses that are applied to electrophysiologically recorded spikes (e.g., estimation of average firing rates, or tuning curves). Here we show, however, that in the presence of noise this approach is often heavily biased. We propose an alternative analysis that a…
University of Michigan（密歇根大学）；University of Western Australia（西澳大利亚大学）
Spiking neural networks can compensate for quantization error by encoding information either in the temporal domain, or by processing discretized quantities in hidden states of higher precision. In theory, a wide dynamic range state-space enables multiple binarized inputs to be accumulated together, thus improving the representational capacity of individual neurons. This may be achieved by increasing the firing threshold, but make it too high and sparse spike activity turns into no spike emission. In this paper, we propose the use of `threshold annealing' as a warm-up method for firing thresho…
Many neurons display bistability - coexistence of two firing modes such as bursting and tonic spiking or tonic spiking and silence. Bistability has been proposed to endow neurons with richer forms of information processing in general and to be involved in short-term memory in particular by allowing a brief signal to elicit long-lasting changes in firing. In this paper, we focus on bistability that allows for a choice between tonic spiking and depolarization block in a wide range of the depolarization levels. We consider the spike-producing currents in two neurons, models of which differ by the…
Synapses change on multiple timescales, ranging from milliseconds to minutes, due to a combination of both short- and long-term plasticity. Here we develop an extension of the common Generalized Linear Model to infer both short- and long-term changes in the coupling between a pre- and post-synaptic neuron based on observed spiking activity. We model short-term synaptic plasticity using additive effects that depend on the presynaptic spike timing, and we model long-term changes in both synaptic weight and baseline firing rate using point process adaptive smoothing. Using simulations, we first s…
The study of balanced networks of excitatory and inhibitory neurons has led to several open questions. On the one hand it is yet unclear whether the asynchronous state observed in the brain is autonomously generated, or if it results from the interplay between external drivings and internal dynamics. It is also not known, which kind of network variabilities will lead to irregular spiking and which to synchronous firing states. Here we show how isolated networks of purely excitatory neurons generically show asynchronous firing whenever a minimal level of structural variability is present togeth…
We study a network of spiking neurons with heterogeneous excitabilities connected via inhibitory delayed pulses. For globally coupled systems the increase of the inhibitory coupling reduces the number of firing neurons by following a Winner Takes All mechanism. For sufficiently large transmission delay we observe the emergence of collective oscillations in the system beyond a critical coupling value. Heterogeneity promotes neural inactivation and asynchronous dynamics and its effect can be counteracted by considering longer time delays. In sparse networks, inhibition has the counterintuitive e…
Research showed that, the information transmitted in biological neurons is encoded in the instants of successive action potentials or their firing rate. In addition to that, in-vivo operation of the neuron makes measurement difficult and thus continuous data collection is restricted. Due to those reasons, classical mean square estimation techniques that are frequently used in neural network training is very difficult to apply. In such situations, point processes and related likelihood methods may be beneficial. In this study, we will present how one can apply certain methods to use the stimulu…
Spontaneous fluctuations and stimulus response are essential features of neural functioning but how they are connected is poorly understood. I derive fluctuation-dissipation relations (FDR) between the spontaneous spike and voltage correlations and the firing rate susceptibility for i) the leaky integrate-and-fire (IF) model with white noise; ii) an IF model with arbitrary voltage dependence, an adaptation current, and correlated noise. The FDRs can be used to derive correlation statistics or to infer the system's response from observations of its spontaneous activity.
At functional scales, cortical behavior results from the complex interplay of a large number of excitable cells operating in noisy environments. Such systems resist to mathematical analysis, and computational neurosciences have largely relied on heuristic partial (and partially justified) macroscopic models, which successfully reproduced a number of relevant phenomena. The relationship between these macroscopic models and the spiking noisy dynamics of the underlying cells has since then been a great endeavor. Based on recent mean-field reductions for such spiking neurons, we present here {a pr…
Neural networks are complex algorithms that loosely model the behaviour of the human brain. They play a significant role in computational neuroscience and artificial intelligence. The next generation of neural network models is based on the spike timing activity of neurons: spiking neural networks (SNNs). However, model parameters in SNNs are difficult to search and optimise. Previous studies using genetic algorithm (GA) optimisation of SNNs were focused mainly on simple, feedforward, or oscillatory networks, but not much work has been done on optimising cortex-like recurrent SNNs. In this wor…
The response of a neural cell to an external stimulus can follow one of the two patterns: Nonresonant neurons monotonously relax to the resting state after excitation while resonant ones show subthreshold oscillations. We investigate how do these subthreshold properties of neurons affect their suprathreshold response. Vice versa we ask: Can we distinguish between both types of neuronal dynamics using suprathreshold spike trains? The dynamics of neurons is given by stochastic FitzHugh-Nagumo and Morris-Lecar models with either having a focus or a node as the stable fixpoint. We determine numeri…
