We extensively explore networks of weakly unbalanced, leaky integrate-and-fire (LIF) neurons for different coupling strength, connectivity, and by varying the degree of refractoriness, as well as the delay in the spike transmission. We find that the neural network does not only exhibit a microscopic (single-neuron) stochastic-like evolution, but also a collective irregular dynamics (CID). Our analysis is based on the computation of a suitable order parameter, typically used to characterize synchronization phenomena and on a detailed scaling analysis (i.e. simulations of different network sizes). As a result, we can conclude that CID is a true thermodynamic phase, intrinsically different from the standard asynchronous regime.
We study the dynamics of identical leaky integrate-and-fire neurons with symmetric non-local coupling. Upon varying control parameters (coupling strength, coupling range, refractory period) we investigate the system's behaviour and highlight the formation of chimera states. We show that the introduction of a refractory period enlarges the parameter region where chimera states appear and affects the chimera multiplicity.
We consider a single Leaky integrate-and-fire neuron stimulated with Poisson process. We develop a method, which allows one to obtain the first passage time probability density function without any additional approximations.
Probability density function of output interspike intervals is found in exact form for leaky integrate and fire neuron stimulated with Poisson stream. The diffusion approximation is not exploited.
We compute the firing rate of a leaky integrate-and-fire (LIF) neuron with stochastic conductance-based inputs in the limit when synaptic decay times are much shorter than the membrane time constant. A comparison of our analytical results to numeric simulations is presented for a range of biophysically-realistic parameters.
We study the synchronization of fully-connected and totally excitatory integrate and fire neural networks in presence of Gaussian white noises. Using a large deviation principle, we prove the stability of the synchronized state under stochastic perturbations. Then, we give a lower bound on the probability of synchronization for networks which are not initially synchronized. This bound shows the robustness of the emergence of synchronization in presence of small stochastic perturbations.
This work reports a transfer function-based approach to characterizing the operation of single neuronal cells in terms of the instantaneous frequency of the input and output signals. We adopt the leaky integrate-and-fire model. The transfer relationship is obtained by performing successive numeric-computational simulations and statistical regressions. Several interesting results are reported, including the identification and characterization of linearity in the transfer relationship, as well as the identification of regions in the parameter space characterized by sharper transfer functions. Th…
We show that the stochastic Morris-Lecar neuron, in a neighborhood of its stable point, can be approximated by a two-dimensional Ornstein-Uhlenbeck (OU) modulation of a constant circular motion. The associated radial OU process is an example of a leaky integrate-and-fire (LIF) model prior to firing. A new model constructed from a radial OU process together with a simple firing mechanism based on detailed Morris-Lecar firing statistics reproduces the Morris-Lecar Interspike Interval (ISI) distribution, and has the computational advantages of a LIF. The result justifies the large amount of atten…
The response properties of excitable systems driven by colored noise are of great interest, but are usually mathematically only accessible via approximations. For this reason, dichotomous noise, a rare example of a colored noise leading often to analytically tractable problems, has been extensively used in the study of stochastic systems. Here, we calculate exact expressions for the power spectrum and the susceptibility of a leaky integrate-and-fire neuron driven by asymmetric dichotomous noise. While our results are in excellent agreement with simulations, they also highlight a limitation of …
One of the most important challenges in mathematical neuroscience is to properly illustrate the stochastic nature of neurons. Among different approaches, the noisy leaky integrate-and-fire and the escape rate models are probably the most popular. These two models are usually chosen to express different noise action over the neural cell. In this paper we investigate the link between the two formalisms in the case of a neuron subject to a time dependent input. To this aim, we introduce a new general stochastic framework. As we shall prove, our general framework entails the two already existing o…
The leaky integrate and fire (LIF) neuron represents standard neuronal model used for numerical simulations. The leakage is implemented in the model as exponential decay of trans-membrane voltage towards its resting value. This makes inevitable the usage of machine floating point numbers in the course of simulation. It is known that machine floating point arithmetic is subjected to small inaccuracies, which prevent from exact comparison of floating point quantities. In particular, it is incorrect to decide whether two separate in time states of a simulated system composed of LIF neurons are ex…
The statistics of the output activity of a neuron during its stimulation by the stream of input impulses that forms the stochastic Poisson process is studied. The leaky integrate-and-fire neuron is considered as a neuron model. A new representation of the probability distribution function of the output interspike interval durations is found. Based on it, the moment-generating function of the probability distribution is calculated explicitly. The latter, according to Curtiss theorem, completely determines the distribution itself. In particular, explicit expressions are derived from the moment-g…
The Nonlinear Noisy Leaky Integrate and Fire neuronal models are mathematical models that describe the activity of neural networks. These models have been studied at a microscopic level, using Stochastic Differential Equations, and at a mesoscopic/macroscopic level, through the mean field limits using Fokker-Planck type equations. The aim of this paper is to improve their understanding, using a numerical study of their particle systems. We analyse in depth the behaviour of the classical and physical solutions of the Stochastic Differential Equations and, we compare it with what is already know…
Providing an analytical treatment to the stochastic feature of neurons' dynamics is one of the current biggest challenges in mathematical biology. The noisy leaky integrate-and-fire model and its associated Fokker-Planck equation are probably the most popular way to deal with neural variability. Another well-known formalism is the escape-rate model: a model giving the probability that a neuron fires at a certain time knowing the time elapsed since its last action potential. This model leads to a so-called age-structured system, a partial differential equation with non-local boundary condition …
Statistical properties of spike trains as well as other neurophysiological data suggest a number of mathematical models of neurons. These models range from entirely descriptive ones to those deduced from the properties of the real neurons. One of them, the diffusion leaky integrate-and-fire neuronal model, which is based on the Ornstein-Uhlenbeck stochastic process that is restricted by an absorbing barrier, can describe a wide range of neuronal activity in terms of its parameters. These parameters are readily associated with known physiological mechanisms. The other model is descriptive, Gamm…
We investigate numerically the collective dynamical behavior of pulse-coupled non-leaky integrate-and-fire-neurons that are arranged on a two-dimensional small-world network. To ensure ongoing activity, we impose a probability for spontaneous firing for each neuron. We study network dynamics evolving from different sets of initial conditions in dependence on coupling strength and rewiring probability. Beside a homogeneous equilibrium state for low coupling strength, we observe different local patterns including cyclic waves, spiral waves, and turbulent-like patterns, which -- depending on netw…
Neural network dynamics emerge from the interaction of spiking cells. One way to formulate the problem is through a theoretical framework inspired by ideas coming from statistical physics, the so-called mean-field theory. In this document, we investigate different issues related to the mean-field description of an excitatory network made up of leaky integrate-and-fire neurons. The description is written in the form a nonlinear partial differential equation which is known to blow up in finite time when the network is strongly connected. We prove that in a moderate coupling regime the equation i…
Tactile afferents such as (RA), and Pacinian (PC) afferents that respond to external stimuli enable complicated actions such as grasping, stroking and identifying an object. To understand the tactile sensation induced by these actions deeply, the activities of the tactile afferents need to be revealed. For this purpose, we develop a computational model for each tactile afferent for vibration stimuli, combining finite element analysis finite element method (FEM) analysis and a leaky integrate-and-fire model that represents the neural characteristics. This computational model can easily estimate…
We present a biophysical approach for the coupling of neural network activity as resulting from proper dipole currents of cortical pyramidal neurons to the electric field in extracellular fluid. Starting from a reduced threecompartment model of a single pyramidal neuron, we derive an observation model for dendritic dipole currents in extracellular space and thereby for the dendritic field potential that contributes to the local field potential of a neural population. This work aligns and satisfies the widespread dipole assumption that is motivated by the "open-field" configuration of the dendr…
We study the collective dynamics of a Leaky Integrate and Fire network in which precise relative phase relationship of spikes among neurons are stored, as attractors of the dynamics, and selectively replayed at differentctime scales. Using an STDP-based learning process, we store in the connectivity several phase-coded spike patterns, and we find that, depending on the excitability of the network, different working regimes are possible, with transient or persistent replay activity induced by a brief signal. We introduce an order parameter to evaluate the similarity between stored and recalled …
Artificial neural networks (ANNs) have been extensively used for the description of problems arising from biological systems and for constructing neuromorphic computing models. The third generation of ANNs, namely, spiking neural networks (SNNs), inspired by biological neurons enable a more realistic mimicry of the human brain. A large class of the problems from these domains is characterized by the necessity to deal with the combination of neurons, spikes and synapses via integrate-and-fire neuron models. Motivated by important applications of the integrate-and-fire of neurons in neuromorphic…
Stochastic integrate-and-fire (IF) neuron models have found widespread applications in computational neuroscience. Here we present results on the white-noise-driven perfect, leaky, and quadratic IF models, focusing on the spectral statistics (power spectra, cross spectra, and coherence functions) in different dynamical regimes (noise-induced and tonic firing regimes with low or moderate noise). We make the models comparable by tuning parameters such that the mean value and the coefficient of variation of the interspike interval match for all of them. We find that, under these conditions, the p…
Friedrich Alexander University Erlangen-Nürnberg (FAU)（埃尔朗根-纽伦堡弗里德里希-亚历山大大学（FAU））；University Hospital Erlangen（埃尔朗根大学医院）；Friedrich-Alexander University Erlangen-Nürnberg (FAU)（埃尔朗根-纽伦堡弗里德里希-亚历山大大学（FAU））
Up to now, modern Machine Learning is mainly based on fitting high dimensional functions to enormous data sets, taking advantage of huge hardware resources. We show that biologically inspired neuron models such as the Leaky-Integrate-and-Fire (LIF) neurons provide novel and efficient ways of information encoding. They can be integrated in Machine Learning models, and are a potential target to improve Machine Learning performance. Thus, we derived simple update-rules for the LIF units from the differential equations, which are easy to numerically integrate. We apply a novel approach to train th…
Several experimental studies show the existence of leader neurons in population bursts of 2D living neural networks. A leader neuron is, basically, a neuron which fires at the beginning of a burst (respectively network spike) more often that we expect by looking at its whole mean neural activity. This means that leader neurons have some burst triggering power beyond a simple statistical effect. In this study, we characterize these leader neuron properties. This naturally leads us to simulate neural 2D networks. To build our simulations, we choose the leaky integrate and fire (lIF) neuron model…
We revisit the dynamics of a prototypical model of balanced activity in networks of spiking neutrons. A detailed investigation of the thermodynamic limit for fixed density of connections (massive coupling) shows that, when inhibition prevails, the asymptotic regime is not asynchronous but rather characterized by a self-sustained irregular, macroscopic (collective) dynamics. So long as the connectivity is massive, this regime is found in many different setups: leaky as well as quadratic integrate-and-fire neurons; large and small coupling strength; weak and strong external currents.
ILIF: Temporal Inhibitory Leaky Integrate-and-Fire Neuron for Overactivation in Spiking Neural Networks
Department of Data Science and AI, Monash University（墨尔本大学数据科学与人工智能系）；School of Computer Science and Engineering, Southeast University（东南大学计算机科学与工程学院）；School of Software, Northeastern University（东北大学软件学院）
We studied the impact of a dynamical threshold on the f-I curve-the relationship between the input and the firing rate of a neuron-in the presence of background synaptic inputs. First, we found that, while the leaky integrate-and-fire model cannot reproduce the f-I curve of a cortical neuron, the leaky integrate-and-fire model with dynamical threshold can reproduce it very well. Second, we found that the dynamical threshold modulates the onset and the asymptotic behavior of the f-I curve. These results suggest that a cortical neuron has an adaptation mechanism and that the dynamical threshold …
The first-passage-time distribution of a leaky integrate-and-fire neuron driven by a characteristically coloured noise is approximated by matching a transient and a steady-state solution of the membrane voltage distribution. These approximations follow from a simple manipulation, made possible by the specific `eigen' colouring of the noise, which allows to express the membrane potential as a Gaussian diffusion process on top of a deterministic exponential movement. Following, the presented method is extended to the case of an arbitrarily coloured noise driving by factoring out the `eigen' nois…
A complex interplay of single-neuron properties and the recurrent network structure shapes the activity of cortical neurons. The single-neuron activity statistics differ in general from the respective population statistics, including spectra and, correspondingly, autocorrelation times. We develop a theory for self-consistent second-order single-neuron statistics in block-structured sparse random networks of spiking neurons. In particular, the theory predicts the neuron-level autocorrelation times, also known as intrinsic timescales, of the neuronal activity. The theory is based on an extension…
Nonlinear Noisy Leaky Integrate and Fire (NNLIF) models for neurons networks can be written as Fokker-Planck-Kolmogorov equations on the probability density of neurons, the main parameters in the model being the connectivity of the network and the noise. We analyse several aspects of the NNLIF model: the number of steady states, a priori estimates, blow-up issues and convergence toward equilibrium in the linear case. In particular, for excitatory networks, blow-up always occurs for initial data concentrated close to the firing potential. These results show how critical is the balance between n…
We develop a method to investigate the effect of noise timescales on the first-passage time of nonlinear oscillators. Using Fredholm theory, we derive an exact integral equation for the mean event rate of a leaky-integrate-and-fire oscillator that receives constant input and temporally correlated noise. Furthermore, we show that Fredholm theory provides a unified framework to determine system scaling behavior for small and large noise timescales. In this framework, the leading order and higher-order asymptotic corrections for slow and fast noise are naturally emerging. We show the scaling beha…
Integrate-and-fire (IF) neurons have found widespread applications in computational neuroscience. Particularly important are stochastic versions of these models where the driving consists of a synaptic input modeled as white Gaussian noise with mean $μ$ and noise intensity $D$. Different IF models have been proposed, the firing statistics of which depends nontrivially on the input parameters $μ$ and $D$. In order to compare these models among each other, one must first specify the correspondence between their parameters. This can be done by determining which set of parameters ($μ$, $D$) of eac…
Randomly-connected networks of integrate-and-fire (IF) neurons are known to display asynchronous irregular (AI) activity states, which resemble the discharge activity recorded in the cerebral cortex of awake animals. However, it is not clear whether such activity states are specific to simple IF models, or if they also exist in networks where neurons are endowed with complex intrinsic properties similar to electrophysiological measurements. Here, we investigate the occurrence of AI states in networks of nonlinear IF neurons, such as the adaptive exponential IF (Brette-Gerstner-Izhikevich) mode…
Networks in the brain consist of different types of neurons. Here we investigate the influence of neuron diversity on the dynamics, phase space structure and computational capabilities of spiking neural networks. We find that already a single neuron of a different type can qualitatively change the network dynamics and that mixed networks may combine the computational capabilities of ones with a single neuron type. We study inhibitory networks of concave leaky (LIF) and convex "anti-leaky" (XIF) integrate-and-fire neurons that generalize irregularly spiking non-chaotic LIF neuron networks. Endo…
A unique feature of neuromorphic computing is that memory is an implicit part of processing through traces of past information in the system's collective dynamics. The extent of memory about past inputs is commonly quantified by the autocorrelation time of collective dynamics. Based on past experimental evidence, a potential explanation for the underlying autocorrelations are close-to-critical fluctuations. Here, we show for self-organized networks of excitatory and inhibitory leaky integrate-and-fire neurons that autocorrelations can originate from emergent bistability upon reducing external …
We present a mathematical analysis of a networks with Integrate-and-Fire neurons and adaptive conductances. Taking into account the realistic fact that the spike time is only known within some \textit{finite} precision, we propose a model where spikes are effective at times multiple of a characteristic time scale $δ$, where $δ$ can be \textit{arbitrary} small (in particular, well beyond the numerical precision). We make a complete mathematical characterization of the model-dynamics and obtain the following results. The asymptotic dynamics is composed by finitely many stable periodic orbits, wh…
We consider a leaky integrate-and-fire neuron with deterministic subthreshold dynamics and a firing threshold that evolves as an Ornstein-Uhlenbeck process. The formulation of this minimal model is motivated by the experimentally observed widespread variation of neural firing thresholds. We show numerically that the mean first passage time can depend non-monotonically on the noise amplitude. For sufficiently large values of the correlation time of the stochastic threshold the mean first passage time is maximal for non-vanishing noise. We provide an explanation for this effect by analytically t…
Analytical expressions are put forward to investigate the forced spiking activity of abstract neuron models such as the driven leaky integrate-and-fire (LIF) model. The method is valid in a wide parameter regime beyond the restraining limits of weak driving (linear response) and/or weak noise. The novel approximation is based on a discrete state Markovian modeling of the full dynamics with time-dependent rates. The scheme yields very good agreement with numerical Langevin and Fokker-Planck simulations of the full non-stationary dynamics for both, the first-passage time statistics and the inter…
When stimulated by a very large number of Poisson-like presynaptic current input spikes, the temporal dynamics of the soma membrane potential $V(t)$ of a leaky integrate-and-fire (LIF) neuron is typically modeled in the diffusion limit and treated as a Ornstein-Uhlenbeck process (OUP). When the potential reaches a threshold value $θ$, $V(t) = θ$, the LIF neuron fires and the membrane potential is reset to a resting value, $V_0 < θ$, and clamped to this value for a specified (non-stochastic) absolute refractory period $T_r \ge 0$, after which the cycle is repeated. The time between firings is g…
We propose a new supervised learning rule for multilayer spiking neural networks (SNNs) that use a form of temporal coding known as rank-order-coding. With this coding scheme, all neurons fire exactly one spike per stimulus, but the firing order carries information. In particular, in the readout layer, the first neuron to fire determines the class of the stimulus. We derive a new learning rule for this sort of network, named S4NN, akin to traditional error backpropagation, yet based on latencies. We show how approximated error gradients can be computed backward in a feedforward network with an…
Pairs of neurons in brain networks often share much of the input they receive from other neurons. Due to essential non-linearities of neuronal dynamics, the consequences for the correlation of the output spike trains are not well understood in the strongly correlated regime. Here we consider two leaky integrate-and-fire neurons with correlated white noise input. We analyze this scenario using a novel non-perturbative approach. Hence our treatment covers both weakly and strongly correlated dynamics, generalizing previous results based on linear response theory.
Noise in spiking neurons is commonly modeled by a noisy input current or by generating output spikes stochastically with a voltage-dependent hazard rate ("escape noise"). While input noise lends itself to modeling biophysical noise processes, the phenomenological escape noise is mathematically more tractable. Using the level-crossing theory for differentiable Gaussian processes, we derive an approximate mapping between colored input noise and escape noise in leaky integrate-and-fire neurons. This mapping requires the first-passage-time (FPT) density of an overdamped Brownian particle driven by…
We investigate a network of integrate-and-fire neurons characterized by a distribution of spiking frequencies. Upon increasing the coupling strength, the model exhibits a transition from an asynchronous regime to a nontrivial collective behavior. At variance with the Kuramoto model, (i) the macroscopic dynamics is irregular even in the thermodynamic limit, and (ii) the microscopic (single-neuron) evolution is linearly stable.
We provide rigorous and exact results characterizing the statistics of spike trains in a network of leaky integrate and fire neurons, where time is discrete and where neurons are submitted to noise, without restriction on the synaptic weights. We show the existence and uniqueness of an invariant measure of Gibbs type and discuss its properties. We also discuss Markovian approximations and relate them to the approaches currently used in computational neuroscience to analyse experimental spike trains statistics.
We analyze the dynamics of networks of spiking neural oscillators. First, we present an exact linear stability theory of the synchronous state for networks of arbitrary connectivity. For general neuron rise functions, stability is determined by multiple operators, for which standard analysis is not suitable. We describe a general non-standard solution to the multi-operator problem. Subsequently, we derive a class of rise functions for which all stability operators become degenerate and standard eigenvalue analysis becomes a suitable tool. Interestingly, this class is found to consist of networ…
Leaky integrate-and-fire (LIF) models are mean-field limits, with a large number of neurons, used to describe neural networks. We consider inhomogeneous networks structured by a connec-tivity parameter (strengths of the synaptic weights) with the effect of processing the input current with different intensities. We first study the properties of the network activity depending on the distribution of synaptic weights and in particular its discrimination capacity. Then, we consider simple learning rules and determine the synaptic weight distribution it generates. We outline the role of noise as a …
The binding neuron model is inspired by numerical simulation of Hodgkin-Huxley-type point neuron, as well as by the leaky integrate-and-fire model. In the binding neuron, the trace of an input is remembered for a fixed period of time after which it disappears completely. This is in the contrast with the above two models, where the postsynaptic potentials decay exponentially and can be forgotten only after triggering. The finiteness of memory in the binding neuron allows one to construct fast recurrent networks for computer modeling. Recently, the finiteness is utilized for exact mathematical d…
The dynamics of neural networks is often characterized by collective behavior and quasi-synchronous events, where a large fraction of neurons fire in short time intervals, separated by uncorrelated firing activity. These global temporal signals are crucial for brain functioning. They strongly depend on the topology of the network and on the fluctuations of the connectivity. We propose a heterogeneous mean--field approach to neural dynamics on random networks, that explicitly preserves the disorder in the topology at growing network sizes, and leads to a set of self-consistent equations. Within…
A recent study on the effect of colored driving noise on the escape from a metastable state derives an analytic expression of the transfer function of the leaky integrate-and-fire neuron model subject to colored noise. Here we present an alternative derivation of the results, taking into account time-dependent boundary conditions explicitly. This systematic approach may facilitate future extensions beyond first order perturbation theory. The analogy of the quantum harmonic oscillator to the LIF neuron model subject to white noise enables a derivation of the well known transfer function simpler…
The seemingly stochastic transient dynamics of neocortical circuits observed in vivo have been hypothesized to represent a signature of ongoing stochastic inference. In vitro neurons, on the other hand, exhibit a highly deterministic response to various types of stimulation. We show that an ensemble of deterministic leaky integrate-and-fire neurons embedded in a spiking noisy environment can attain the correct firing statistics in order to sample from a well-defined target distribution. We provide an analytical derivation of the activation function on the single cell level; for recurrent netwo…
We review the use of mean field theory for describing the dynamics of dense, randomly connected cortical circuits. For a simple network of excitatory and inhibitory leaky integrate-and-fire neurons, we can show how the firing irregularity, as measured by the Fano factor, increases with the strength of the synapses in the network and with the value to which the membrane potential is reset after a spike. Generalizing the model to include conductance-based synapses gives insight into the connection between the firing statistics and the high-conductance state observed experimentally in visual cort…
Finite-sized populations of spiking elements are fundamental to brain function, but also used in many areas of physics. Here we present a theory of the dynamics of finite-sized populations of spiking units, based on a quasi-renewal description of neurons with adaptation. We derive an integral equation with colored noise that governs the stochastic dynamics of the population activity in response to time-dependent stimulation and calculate the spectral density in the asynchronous state. We show that systems of coupled populations with adaptation can generate a frequency band in which sensory inf…
In machine learning, error back-propagation in multi-layer neural networks (deep learning) has been impressively successful in supervised and reinforcement learning tasks. As a model for learning in the brain, however, deep learning has long been regarded as implausible, since it relies in its basic form on a non-local plasticity rule. To overcome this problem, energy-based models with local contrastive Hebbian learning were proposed and tested on a classification task with networks of rate neurons. We extended this work by implementing and testing such a model with networks of leaky integrate…
We study the spike statistics of neurons in a network with dynamically balanced excitation and inhibition. Our model, intended to represent a generic cortical column, comprises randomly connected excitatory and inhibitory leaky integrate-and-fire neurons, driven by excitatory input from an external population. The high connectivity permits a mean-field description in which synaptic currents can be treated as Gaussian noise, the mean and autocorrelation function of which are calculated self-consistently from the firing statistics of single model neurons. Within this description, we find that th…
