Runtime Construction of Large-Scale Spiking Neuronal Network Models on GPU Devices
Bruno Golosio, Jose Villamar, Gianmarco Tiddia, Elena Pastorelli, Jonas Stapmanns, Viviana Fanti, Pier Stanislao Paolucci, Abigail Morrison, Johanna Senk
Simulation speed matters for neuroscientific research: this includes not only how quickly the simulated model time of a large-scale spiking neuronal network progresses, but also how long it takes to instantiate the network model in computer memory. On the hardware side, acceleration via highly parallel GPUs is being increasingly utilized. On the software side, code generation approaches ensure highly optimized code, at the expense of repeated code regeneration and recompilation after modifications to the network model. Aiming for a greater flexibility with respect to iterative model changes, here we propose a new method for creating network connections interactively, dynamically, and directly in GPU memory through a set of commonly used high-level connection rules. We validate the simulation performance with both consumer and data center GPUs on two neuroscientifically relevant models: a cortical microcircuit of about 77,000 leaky-integrate-and-fire neuron models and 300 million static synapses, and a two-population network recurrently connected using a variety of connection rules. With our proposed ad hoc network instantiation, both network construction and simulation times are comparable or shorter than those obtained with other state-of-the-art simulation technologies, while still meeting the flexibility demands of explorative network modeling.
Harmonic fractal transformation for modeling complex neuronal effects: from bursting and noise shaping to waveform sensitivity and noise-induced subthreshold spiking
Neuronal spiking exhibits an exquisite combination of modulation and robustness properties, rarely matched in artificial systems. We exploit the particular interconnection structure of conductance based models to investigate this remarkable property. We find that much of neuronal modulation and robustness can be explained by separating the total transmembrane current into three different components corresponding to the three time scales of neuronal bursting. Each equivalent current aggregates many ionic contributions into an equivalent voltage-dependent conductance, which defines a key modulat…
We establish a general linear response relation for spiking neuronal networks, based on chains with unbounded memory. This relation allows us to predict the influence of a weak amplitude time-dependent external stimuli on spatio-temporal spike correlations, from the spontaneous statistics (without stimulus) in a general context where the memory in spike dynamics can extend arbitrarily far in the past. Using this approach, we show how linear response is explicitly related to neuronal dynamics with an example, the gIF model, introduced by M. Rudolph and A. Destexhe. This example illustrates the …
Neuro-inspired models and systems have great potential for applications in unconventional computing. Often, the mechanisms of biological neurons are modeled or mimicked in simulated or physical systems in an attempt to harness some of the computational power of the brain. However, the biological mechanisms at play in neural systems are complicated and challenging to capture and engineer; thus, it can be simpler to turn to a data-driven approach to transfer features of neural behavior to artificial substrates. In the present study, we used an evolutionary algorithm (EA) to produce spiking neura…
At functional scales, cortical behavior results from the complex interplay of a large number of excitable cells operating in noisy environments. Such systems resist to mathematical analysis, and computational neurosciences have largely relied on heuristic partial (and partially justified) macroscopic models, which successfully reproduced a number of relevant phenomena. The relationship between these macroscopic models and the spiking noisy dynamics of the underlying cells has since then been a great endeavor. Based on recent mean-field reductions for such spiking neurons, we present here {a pr…
We examine whether a single biophysical cortical circuit model can explain both spiking and perceptual variability. We consider perceptual rivalry, which provides a window into intrinsic neural processing since neural activity in some brain areas is correlated to the alternating perception rather than the constant ambiguous stimulus. The prevalent theory for spiking variability is a chaotic attractor called the balanced state; whereas, the source of perceptual variability is an open question. We present a dynamical model with a chaotic attractor that explains both spiking and perceptual variab…
The influence of temperature on neuronal excitability is studied by numerical simulations on the spiking threshold characteristics of bushy cells in cochlear nucleus periodically stimulated by synaptic currents. The results reveal that there is a cut-off frequency for the spiking of bushy cell in a specific temperature environment, corresponding to the existence of a critical temperature for the neuron to respond with real spikes to the synaptic stimulus of a given frequency, due to the finiteness of spike width. An optimal temperature range for neuronal spiking is also found for a specific st…
Rapid action potential generation --- spiking --- and alternating intervals of spiking and quiescence --- bursting --- are two dynamic patterns observed in neuronal activity. In computational models of neuronal systems, the transition from spiking to bursting often exhibits complex bifurcation structure. One type of transition involves the torus canard, which was originally observed in a simple biophysical model of a Purkinje cell. In this article, we expand on that original result by showing that torus canards arise in a broad array of well-known computational neuronal models with three diffe…
Investigating the spectral properties of the neural covariates that underlie spiking activity is an important problem in systems neuroscience, as it allows to study the role of brain rhythms in cognitive functions. While the spectral estimation of continuous time-series is a well-established domain, computing the spectral representation of these neural covariates from spiking data sets forth various challenges due to the intrinsic non-linearities involved. In this paper, we address this problem by proposing a variant of the multitaper method specifically tailored for point process data. To thi…
Quantification of neuronal correlations in neuron populations helps us to understand neural coding rules. Such quantification could also reveal how neurons encode information in normal and disease conditions like Alzheimer's and Parkinson's. While neurons communicate with each other by transmitting spikes, there would be a change in calcium concentration within the neurons inherently. Accordingly, there would be correlations in calcium spike trains and they could have heterogeneous memory structures. In this context, estimation of mutual information rate in calcium spike trains assumes primary…
Computational modeling is becoming a widely used methodology in modern neuroscience. However, as the complexity of the phenomena under study increases, the analysis of the results emerging from the simulations concomitantly becomes more challenging. In particular, the configuration and validation of brain circuits involving learning often require the processing of large amounts of action potentials and their comparison to the stimulation being presented to the input of the system. In this study we present a systematic work-flow for the configuration of spiking-neuronal-network-based learning s…
Although there is increasing evidence of criticality in the brain, the processes that guide neuronal networks to reach or maintain criticality remain unclear. The present research examines the role of neuronal gain plasticity in time-series of simulated neuronal networks composed of integrate-and-fire stochastic spiking neurons, and the utility of fractal methods in assessing network criticality. Simulated time-series were derived from a network model of fully connected discrete-time stochastic excitable neurons. Monofractal and multifractal analyses were applied to neuronal gain time-series. …
A common approach to interpreting spiking activity is based on identifying the firing fields---regions in physical or configuration spaces that elicit responses of neurons. Common examples include hippocampal place cells that fire at preferred locations in the navigated environment, head direction cells that fire at preferred orientations of the animal's head, view cells that respond to preferred spots in the visual field, etc. In all these cases, firing fields were discovered empirically, by trial and error. We argue that the existence and a number of properties of the firing fields can be es…
We study the effect of intrinsic heterogeneity on the activity of a population of leaky integrate-and-fire neurons. By rescaling the dynamical equation, we derive mathematical relations between multiple neuronal parameters and a fluctuating input noise. To this end, common input to heterogeneous neurons is conceived as an identical noise with neuron-specific mean and variance. As a consequence, the neuronal output rates can differ considerably, and their relative spike timing becomes desynchronized. This theory can quantitatively explain some recent experimental findings.
Spiking neural networks (SNNs) communicate through the all-or-none spiking activity of neurons. However, fitting the large number of SNN model parameters to observed neural activity patterns, for example, in biological experiments, remains a challenge. Previous work using genetic algorithm (GA) optimisation on a specific efficient SNN model, using the Izhikevich neuronal model, was limited to a single parameter and objective. This work applied a version of GA, called non-dominated sorting GA (NSGA-III), to demonstrate the feasibility of performing multi-objective optimisation on the same SNN, …
When neural circuits learn to perform a task, it is often the case that there are many sets of synaptic connections that are consistent with the task. However, only a small number of possible solutions are robust to noise in the input and are capable of generalizing their performance of the task to new inputs. Finding such good solutions is an important goal of learning systems in general and neuronal circuits in particular. For systems operating with static inputs and outputs, a well known approach to the problem is the large margin methods such as Support Vector Machines (SVM). By maximizing…
Neural networks are complex algorithms that loosely model the behaviour of the human brain. They play a significant role in computational neuroscience and artificial intelligence. The next generation of neural network models is based on the spike timing activity of neurons: spiking neural networks (SNNs). However, model parameters in SNNs are difficult to search and optimise. Previous studies using genetic algorithm (GA) optimisation of SNNs were focused mainly on simple, feedforward, or oscillatory networks, but not much work has been done on optimising cortex-like recurrent SNNs. In this wor…
We analyze the collective dynamics of hierarchically structured networks of densely connected spiking neurons. These networks of sub-networks may represent interactions between cell assemblies or different nuclei in the brain. The dynamical activity pattern that results from these interactions depends on the strength of synaptic coupling between them. Importantly, the overall dynamics of a brain region in the absence of external input, so called ongoing brain activity, has been attributed to the dynamics of such interactions. In our study, two different network scenarios are considered: a syst…
Many types of neurons exhibit spike rate adaptation, mediated by intrinsic slow $\mathrm{K}^+$-currents, which effectively inhibit neuronal responses. How these adaptation currents change the relationship between in-vivo like fluctuating synaptic input, spike rate output and the spike train statistics, however, is not well understood. In this computational study we show that an adaptation current which primarily depends on the subthreshold membrane voltage changes the neuronal input-output relationship (I-O curve) subtractively, thereby increasing the response threshold. A spike-dependent adap…
We numerically investigate the influence of intrinsic channel noise on the dynamical response of delay-coupling in neuronal systems. The stochastic dynamics of the spiking is modeled within a stochastic modification of the standard Hodgkin-Huxley model wherein the delay-coupling accounts for the finite propagation time of an action potential along the neuronal axon. We quantify this delay-coupling of the Pyragas-type in terms of the difference between corresponding presynaptic and postsynaptic membrane potentials. For an elementary neuronal network consisting of two coupled neurons we detect c…
Cortical neural circuits display highly irregular spiking in individual neurons but variably sized collective firing, oscillations and critical avalanches at the population level, all of which have functional importance for information processing. Theoretically, the balance of excitation and inhibition inputs is thought to account for spiking irregularity and critical avalanches may originate from an underlying phase transition. However, the theoretical reconciliation of these multilevel dynamic aspects in neural circuits remains an open question. Herein, we study excitation-inhibition (E-I) b…
We present a modified library-based method for simulating the Hodgkin-Huxley (HH) neuronal networks. By pre-computing a high resolution data library during the interval of an action potential (spike), we can avoid evolving the HH equations during the spike and can use a large time step to raise efficiency. The library method can stably achieve at most 10 times of speedup compared with the regular Runge-Kutta method while capturing most statistical properties of HH neurons like the distribution of spikes which data is widely used in the statistical analysis like transfer entropy and Granger cau…
An essential step toward understanding neural circuits is linking their structure and their dynamics. In general, this relationship can be almost arbitrarily complex. Recent theoretical work has, however, begun to identify some broad principles underlying collective spiking activity in neural circuits. The first is that local features of network connectivity can be surprisingly effective in predicting global statistics of activity across a network. The second is that, for the important case of large networks with excitatory-inhibitory balance, correlated spiking persists or vanishes depending …
Neurons in cortical circuits exhibit coordinated spiking activity, and can produce correlated synchronous spikes during behavior and cognition. We recently developed a method for estimating the dynamics of correlated ensemble activity by combining a model of simultaneous neuronal interactions (e.g., a spin-glass model) with a state-space method (Shimazaki et al. 2012 PLoS Comput Biol 8 e1002385). This method allows us to estimate stimulus-evoked dynamics of neuronal interactions which is reproducible in repeated trials under identical experimental conditions. However, the method may not be sui…
Recent remarkable advances in the experimental techniques have provided a background for inferring neuronal couplings from point process data that includes a great number of neurons. Here, we propose a systematic procedure for pre- and post-processing generic point process data in an objective manner, to handle data in the framework of a binary simple statistical model, the Ising or generalized McCulloch--Pitts model. The procedure involves two steps: (1) determining time-bin size for transforming the point-process data into discrete-time binary data and (2) screening relevant couplings from t…
We study an abstracted model of neuronal activity via numerical simulation, and report spatiotemporal pattern formation and critical like dynamics. A population of pulse coupled, discretised, relaxation oscillators is simulated over networks with varying edge density and spatial embedded ness. For intermediate edge density and sufficiently strong spatial embeddedness, we observe a novel spatiotemporal pattern in the field of oscillator phases, visually resembling the surface of a frothing liquid. Increasing the edge density results in critical dynamics, with the distribution of neuronal avalan…
Chemical and electrical synapses shape the dynamics of neuronal networks. Numerous theoretical studies have investigated how each of these types of synapses contributes to the generation of neuronal oscillations, but their combined effect is less understood. This limitation is further magnified by the impossibility of traditional neuronal mean-field models ---also known as firing rate models, or firing rate equations--- to account for electrical synapses. Here we introduce a novel firing rate model that exactly describes the mean field dynamics of heterogeneous populations of quadratic integra…
Determining how synaptic coupling within and between regions is modulated during sensory processing is an important topic in neuroscience. Electrophysiological recordings provide detailed information about neural spiking but have traditionally been confined to a particular region or layer of cortex. Here we develop new theoretical methods to study interactions between and within two brain regions, based on experimental measurements of spiking activity simultaneously recorded from the two regions. By systematically comparing experimentally-obtained spiking statistics to (efficiently computed) m…
Many neurons display bistability - coexistence of two firing modes such as bursting and tonic spiking or tonic spiking and silence. Bistability has been proposed to endow neurons with richer forms of information processing in general and to be involved in short-term memory in particular by allowing a brief signal to elicit long-lasting changes in firing. In this paper, we focus on bistability that allows for a choice between tonic spiking and depolarization block in a wide range of the depolarization levels. We consider the spike-producing currents in two neurons, models of which differ by the…
The aim of this paper is to investigate various information-theoretic measures, including entropy, mutual information, and some systematic measures that based on mutual information, for a class of structured spiking neuronal network. In order to analyze and compute these information-theoretic measures for large networks, we coarse-grained the data by ignoring the order of spikes that fall into the same small time bin. The resultant coarse-grained entropy mainly capture the information contained in the rhythm produced by a local population of the network. We first proved that these information …
A major goal of neuroscience, statistical physics and nonlinear dynamics is to understand how brain function arises from the collective dynamics of networks of spiking neurons. This challenge has been chiefly addressed through large-scale numerical simulations. Alternatively, researchers have formulated mean-field theories to gain insight into macroscopic states of large neuronal networks in terms of the collective firing activity of the neurons, or the firing rate. However, these theories have not succeeded in establishing an exact correspondence between the firing rate of the network and the…
The effect of environmental temperature on neuronal spiking behaviors is investigated by numerically simulating the temperature dependence of spiking threshold of the Hodgkin-Huxley neuron subject to synaptic stimulus. We find that the spiking threshold exhibits a global minimum in a "comfortable temperature" range where spike initiation needs weakest synaptic strength, indicating the occurrence of optimal use of synaptic transmission in neural system. We further explore the biophysical origin of this phenomenon in ion channel gating kinetics and also discuss its possible biological relevance …
Molecule-based devices are envisioned to complement silicon devices by providing new functions or already existing functions at a simpler process level and at a lower cost by virtue of their self-organization capabilities, moreover, they are not bound to von Neuman architecture and this may open the way to other architectural paradigms. Here we demonstrate a device made of conjugated molecules and metal nanoparticles (NPs) which behaves as a spiking synapse suitable for integration in neural network architectures. We demonstrate that this device exhibits the main behavior of a biological synap…
The detection of neuronal communities is addressed with basis on two important concepts from neuroscience: facilitation of neuronal firing and nearly simultaneous beginning of activation of sets of neurons. More specifically, integrate-and-fire complex neuronal networks are activated at each of their nodes, and the dissemination of activation is monitored. As the activation received by each neuron accumulates, its firing gets facilitated. The time it takes for each neuron, other than the source, to receive the first non-zero input (beginning activation time) and the time for it to produce the …
Although recent neurophysiological experiments suggest that synchronous neural activity is involved in some perceptual and cognitive processes, the functional role of such coherent neuronal behavior is not well understood. As a first step in clarifying this role, we investigate how the temporal coherence of certain neuronal activity affects the activity pattern in a neural network. Using a simple network of leaky integrate-and-fire neurons, we study the effects of synchronized incoming spikes on the functioning of two mechanisms typically used in model neural systems, winner-take-all competiti…
An appealing new principle for neural population codes is that correlations among neurons organize neural activity patterns into a discrete set of clusters, which can each be viewed as a noise-robust population "codeword". Previous studies assumed that these codewords corresponded geometrically with local peaks in the probability landscape of neural population responses. Here, we analyze multiple datasets of the responses of ~150 retinal ganglion cells and show that local probability peaks are absent under broad, non-repeated stimulus ensembles, which are characteristic of natural behavior. Ho…
Motifs are patterns of subgraphs of complex networks. We studied the impact of such patterns of connectivity on the level of correlated, or synchronized, spiking activity among pairs of cells in a recurrent network model of integrate and fire neurons. For a range of network architectures, we find that the pairwise correlation coefficients, averaged across the network, can be closely approximated using only three statistics of network connectivity. These are the overall network connection probability and the frequencies of two second-order motifs: diverging motifs, in which one cell provides in…
Dynamics and function of neuronal networks are determined by their synaptic connectivity. Current experimental methods to analyze synaptic network structure on the cellular level, however, cover only small fractions of functional neuronal circuits, typically without a simultaneous record of neuronal spiking activity. Here we present a method for the reconstruction of large recurrent neuronal networks from thousands of parallel spike train recordings. We employ maximum likelihood estimation of a generalized linear model of the spiking activity in continuous time. For this model the point proces…
The mathematical theory of pattern formation in electrically coupled networks of excitable neurons forced by small noise is presented in this work. Using the Freidlin-Wentzell large deviation theory for randomly perturbed dynamical systems and the elements of the algebraic graph theory, we identify and analyze the main regimes in the network dynamics in terms of the key control parameters: excitability, coupling strength, and network topology. The analysis reveals the geometry of spontaneous dynamics in electrically coupled network. Specifically, we show that the location of the minima of a ce…
One of the challenges of modern neuroscience is creating a "living computer" based on neural networks grown in vitro. Such an artificial device is supposed to perform neurocomputational tasks and interact with the environment when embodied in a robot. Recent studies have identified the most critical challenge, the search for a neural network architecture to implement associative learning. This work proposes a model of modular architecture with spiking neural networks connected by unidirectional couplings. We show that the model enables training a neuro-robot according to Pavlovian conditioning…
In studies of the visual system as well as in computer vision, the focus is often on contrast edges. However, the primate visual system contains a large number of cells that are insensitive to spatial contrast and, instead, respond to uniform homogeneous illumination of their visual field. The purpose of this information remains unclear. Here, we propose a mechanism that detects feature homogeneity in visual areas, based on latency coding and spike time coincidence, in a purely feed-forward and therefore rapid manner. We demonstrate how homogeneity information can interact with information on …
