During development, the mammalian brain differentiates into specialized regions with distinct functional abilities. While many factors contribute to functional specialization, we explore the effect of neuronal density on the development of neuronal interactions in vitro. Two types of cortical networks, dense and sparse, with 50,000 and 12,000 total cells respectively, are studied. Activation graphs that represent pairwise neuronal interactions are constructed using a competitive first response model. These graphs reveal that, during development in vitro, dense networks form activation connections earlier than sparse networks. Link entropy analysis of dense net- work activation graphs suggests that the majority of connections between electrodes are reciprocal in nature. Information theoretic measures reveal that early functional information interactions (among 3 cells) are synergetic in both dense and sparse networks. However, during later stages of development, previously synergetic relationships become primarily redundant in dense, but not in sparse networks. Large link entropy values in the activation graph are related to the domination of redundant ensembles in late stages of development in dense networks. Results demonstrate differences between dense and sparse networks in terms of informational groups, pairwise relationships, and activation graphs. These differences suggest that variations in cell density may result in different functional specialization of nervous system tissue in vivo.
Correlated electrical activity in neurons is a prominent characteristic of cortical microcircuits. Despite a growing amount of evidence concerning both spike-count and subthreshold membrane potential pairwise correlations, little is known about how different types of cortical neurons convert correlated inputs into correlated outputs. We studied pyramidal neurons and two classes of GABAergic interneurons of layer 5 in neocortical brain slices obtained from rats of both sexes, and we stimulated them with biophysically realistic correlated inputs, generated using dynamic clamp. We found that the …
Top-down feedback in cortex is critical for guiding sensory processing, which has prominently been formalized in the theory of hierarchical predictive coding (hPC). However, experimental evidence for error units, which are central to the theory, is inconclusive, and it remains unclear how hPC can be implemented with spiking neurons. To address this, we connect hPC to existing work on efficient coding in balanced networks with lateral inhibition, and predictive computation at apical dendrites. Together, this work points to an efficient implementation of hPC with spiking neurons, where predictio…
In vivo, neurons establish functional connections and preserve information along their synaptic pathways from one information processing stage to the next in a very efficient manner. Paired spiking (PS) enhancement plays a key role by acting as a temporal filter that deletes less informative spikes. We analyzed the spontaneous neural activity evolution in a hippocampal and a cortical network over several weeks exploring whether the same PS coding mechanism appears in neuronal cultures as well. We show that self-organized neural in vitro networks not only develop characteristic bursting activit…
Complex systems are typically characterized as an intermediate situation between a complete regular structure and a random system. Brain signals can be studied as a striking example of such systems: cortical states can range from highly synchronous and ordered neuronal activity (with higher spiking variability) to desynchronized and disordered regimes (with lower spiking variability). It has been recently shown, by testing independent signatures of criticality, that a phase transition occurs in a cortical state of intermediate spiking variability. Here, we use a symbolic information approach t…
Being permanently confronted with an uncertain world, brains have faced evolutionary pressure to represent this uncertainty in order to respond appropriately. Often, this requires visiting multiple interpretations of the available information or multiple solutions to an encountered problem. This gives rise to the so-called mixing problem: since all of these "valid" states represent powerful attractors, but between themselves can be very dissimilar, switching between such states can be difficult. We propose that cortical oscillations can be effectively used to overcome this challenge. By acting…
In this work we study the excitatory AMPA, and NMDA, and inhibitory GABAA receptor mediated dynamical changes in neuronal networks of neonatal rat cortex in vitro. Extracellular network-wide activity was recorded with 59 planar electrodes simultaneously under different pharmacological conditions. We analyzed the changes of overall network activity and network-wide burst frequency between baseline and AMPA receptor (AMPA-R) or NMDA receptor (NMDA-R) driven activity, as well as between the latter states and disinhibited activity. Additionally, spatiotemporal structures of pharmacologically modif…
We apply an information theoretic treatment of action potential time series measured with microelectrode arrays to estimate the connectivity of mammalian neuronal cell assemblies grown {\it in vitro}. We infer connectivity between two neurons via the measurement of the mutual information between their spike trains. In addition we measure higher point multi-informations between any two spike trains conditional on the activity of a third cell, as a means to identify and distinguish classes of functional connectivity among three neurons. The use of a conditional three-cell measure removes some in…
Neural population equations such as neural mass or field models are widely used to study brain activity on a large scale. However, the relation of these models to the properties of single neurons is unclear. Here we derive an equation for several interacting populations at the mesoscopic scale starting from a microscopic model of randomly connected generalized integrate-and-fire neuron models. Each population consists of 50 -- 2000 neurons of the same type but different populations account for different neuron types. The stochastic population equations that we find reveal how spike-history eff…
The brain exhibits capabilities of fast incremental learning from few noisy examples, as well as the ability to associate similar memories in autonomously-created categories and to combine contextual hints with sensory perceptions. Together with sleep, these mechanisms are thought to be key components of many high-level cognitive functions. Yet, little is known about the underlying processes and the specific roles of different brain states. In this work, we exploited the combination of context and perception in a thalamo-cortical model based on a soft winner-take-all circuit of excitatory and …
Neuro-inspired models and systems have great potential for applications in unconventional computing. Often, the mechanisms of biological neurons are modeled or mimicked in simulated or physical systems in an attempt to harness some of the computational power of the brain. However, the biological mechanisms at play in neural systems are complicated and challenging to capture and engineer; thus, it can be simpler to turn to a data-driven approach to transfer features of neural behavior to artificial substrates. In the present study, we used an evolutionary algorithm (EA) to produce spiking neura…
Spiking neural networks (SNNs) communicate through the all-or-none spiking activity of neurons. However, fitting the large number of SNN model parameters to observed neural activity patterns, for example, in biological experiments, remains a challenge. Previous work using genetic algorithm (GA) optimisation on a specific efficient SNN model, using the Izhikevich neuronal model, was limited to a single parameter and objective. This work applied a version of GA, called non-dominated sorting GA (NSGA-III), to demonstrate the feasibility of performing multi-objective optimisation on the same SNN, …
Sleep is essential for learning and cognition, but the mechanisms by which it stabilizes learning, supports creativity, and manages the energy consumption of networks engaged in post-sleep task have not been yet modelled. During sleep, the brain cycles between non-rapid eye movement (NREM), a mainly unconscious state characterized by collective oscillations, and rapid eye movement (REM), associated with the integrated experience of dreaming. We propose a biologically grounded two-area thalamo-cortical plastic spiking neural network model and investigate the role of NREM - REM cycles on its awa…
We report on simultaneous recordings from cells in all layers of visual cortex and models developed to capture the higher order structure of population spiking activity. Specifically, we use Ising, Restricted Boltzmann Machine (RBM) and semi-Restricted Boltzmann Machine (sRBM) models to reveal laminar patterns of activity. While the Ising model describes only pairwise couplings, the RBM and sRBM capture higher-order dependencies using hidden units. Applied to 32- channel polytrode data recorded from cat visual cortex, the higher-order mod- els discover functional connectivity preferentially li…
Decoding finger velocity from cortical spike trains with recurrent spiking neural networks
Qiushi Academy for Advanced Studies, Zhejiang University（浙江大学求是高等研究院）；College of Biomedical Engineering and Instrument Science, Zhejiang University（浙江大学生物医学工程与仪器科学学院）；Friedrich Miescher Institute for Biomedical Research（弗里德里希·米歇尔生物医学研究所）；Faculty of Science, University of Basel（巴塞尔大学理学院）；China Nanhu Academy of Electronics and Information Technology (CNAEIT)（中国南湖电子信息技术研究院）
Cell assemblies manipulation by optogenetics is pivotal to advance neuroscience and neuroengineering. In in vivo applications, photostimulation often broadly addresses a population of cells simultaneously, leading to feed-forward and to reverberating responses in recurrent microcircuits. The former arise from direct activation of targets downstream, and are straightforward to interpret. The latter are consequence of feedback connectivity and may reflect a variety of time-scales and complex dynamical properties. We investigated wide-field photostimulation in cortical networks in vitro, employin…
Over the past decade there has been a growing interest in the development of parallel hardware systems for simulating large-scale networks of spiking neurons. Compared to other highly-parallel systems, GPU-accelerated solutions have the advantage of a relatively low cost and a great versatility, thanks also to the possibility of using the CUDA-C/C++ programming languages. NeuronGPU is a GPU library for large-scale simulations of spiking neural network models, written in the C++ and CUDA-C++ programming languages, based on a novel spike-delivery algorithm. This library includes simple LIF (leak…
The repertoire of neural activity patterns that a cortical network can produce constrains the network's ability to transfer and process information. Here, we measured activity patterns obtained from multi-site local field potential (LFP) recordings in cortex cultures, urethane anesthetized rats, and awake macaque monkeys. First, we quantified the information capacity of the pattern repertoire of ongoing and stimulus-evoked activity using Shannon entropy. Next, we quantified the efficacy of information transmission between stimulus and response using mutual information. By systematically changi…
Qiushi Academy for Advanced Studies, Zhejiang University（浙江大学求是高等研究院）；College of Biomedical Engineering and Instrument Science, Zhejiang University（浙江大学生物医学工程与仪器科学学院）；College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；China Nanhu Academy of Electronics and Information Technology (CNAEIT)（中国电子科技南湖研究院）
Despite its better bio-plausibility, goal-driven spiking neural network (SNN) has not achieved applicable performance for classifying biological spike trains, and showed little bio-functional similarities compared to traditional artificial neural networks. In this study, we proposed the motorSRNN, a recurrent SNN topologically inspired by the neural motor circuit of primates. By employing the motorSRNN in decoding spike trains from the primary motor cortex of monkeys, we achieved a good balance between classification accuracy and energy consumption. The motorSRNN communicated with the input by…
Synthetic reverberating activity patterns are experimentally generated by stimulation of a subset of neurons embedded in a spontaneously active network of cortical cells in-vitro. The neurons are artificially connected by means of conditional stimulation matrix, forming a synthetic local circuit with a predefined programmable connectivity and time-delays. Possible uses of this experimental design are demonstrated, analyzing the sensitivity of these deterministic activity patterns to transmission delays and to the nature of ongoing network dynamics.
We present a new statistical method to analyze multichannel steady-state local field potentials (LFP) recorded within different sensory cortices of different rodent species. Our spatiotemporal multi-dimensional cluster statistics (MCS) method enables statistical analyzing and comparing clusters of data points in n-dimensional space. We demonstrate that using this approach stimulus-specific attractor-like spatiotemporal activity patterns can be detected and be significantly different from each other during stimulation with long-lasting stimuli. Our method may be applied to other types of multic…
In vitro and in vivo spiking activity clearly differ. Whereas networks in vitro develop strong bursts separated by periods of very little spiking activity, in vivo cortical networks show continuous activity. This is puzzling considering that both networks presumably share similar single-neuron dynamics and plasticity rules. We propose that the defining difference between in vitro and in vivo dynamics is the strength of external input. In vitro, networks are virtually isolated, whereas in vivo every brain area receives continuous input. We analyze a model of spiking neurons in which the input s…
Hybrid Spiking Neural Networks for Low-Power Intra-Cortical Brain-Machine Interfaces
Karlsruhe Institute of Technology, Karlsruhe, Germany（卡尔斯鲁厄理工大学）；FZI Research Center for Information Technology, Karlsruhe, Germany（弗劳恩霍夫信息与通信技术研究中心）；Infineon Technologies, Dresden, Germany（英飞凌科技）
When brain signals are recorded in an electroencephalogram or some similar large-scale record of brain activity, oscillatory patterns are typically observed that are thought to reflect the aggregate electrical activity of the underlying neuronal ensemble. Although it now seems that such patterns participate in feedback loops both temporally with the neurons' spikes and spatially with other brain regions, the mechanisms that might explain the existence of such loops have remained essentially unknown. Here we present a theoretical study of these issues on a cortical model we introduced earlier […
The dominant modeling framework for understanding cortical computations are heuristic firing rate models. Despite their success, these models fall short to capture spike synchronization effects, to link to biophysical parameters and to describe finite-size fluctuations. In this opinion article, we propose that the refractory density method (RDM), also known as age-structured population dynamics or quasi-renewal theory, yields a powerful theoretical framework to build rate-based models for mesoscopic neural populations from realistic neuron dynamics at the microscopic level. We review recent ad…
One of the challenges of modern neuroscience is creating a "living computer" based on neural networks grown in vitro. Such an artificial device is supposed to perform neurocomputational tasks and interact with the environment when embodied in a robot. Recent studies have identified the most critical challenge, the search for a neural network architecture to implement associative learning. This work proposes a model of modular architecture with spiking neural networks connected by unidirectional couplings. We show that the model enables training a neuro-robot according to Pavlovian conditioning…
There are indications that for optimizing neural computation, neural networks - including the brain - operate at criticality. Previous approaches have, however, used diverse fingerprints of criticality, leaving open the question whether they refer to a unique critical point or whether there could be several. Using a recurrent spiking neural network as the model, we demonstrate that avalanche criticality does not necessarily lie at the dynamical edge-of-chaos and that therefore, the different fingerprints indicate distinct phenomena with an as yet unclarified relationship.
Cortical neural circuits display highly irregular spiking in individual neurons but variably sized collective firing, oscillations and critical avalanches at the population level, all of which have functional importance for information processing. Theoretically, the balance of excitation and inhibition inputs is thought to account for spiking irregularity and critical avalanches may originate from an underlying phase transition. However, the theoretical reconciliation of these multilevel dynamic aspects in neural circuits remains an open question. Herein, we study excitation-inhibition (E-I) b…
Sparse coding algorithms trained on natural images can accurately predict the features that excite visual cortical neurons, but it is not known whether such codes can be learned using biologically realistic plasticity rules. We have developed a biophysically motivated spiking network, relying solely on synaptically local information, that can predict the full diversity of V1 simple cell receptive field shapes when trained on natural images. This represents the first demonstration that sparse coding principles, operating within the constraints imposed by cortical architecture, can successfully …
Cortical spreading depression (CSD) is a slow-moving ionic and metabolic disturbance that propagates in cortical brain tissue. In addition to massive cellular depolarization, CSD also involves significant changes in perfusion and metabolism -- aspects of CSD that had not been modeled and are important to traumatic brain injury, subarachnoid hemorrhage, stroke, and migraine. In this study, we develop a mathematical model for CSD where we focus on modeling the features essential to understanding the implications of neurovascular coupling during CSD. In our model, the sodium-potassium--ATPase, ma…
Neural networks are complex algorithms that loosely model the behaviour of the human brain. They play a significant role in computational neuroscience and artificial intelligence. The next generation of neural network models is based on the spike timing activity of neurons: spiking neural networks (SNNs). However, model parameters in SNNs are difficult to search and optimise. Previous studies using genetic algorithm (GA) optimisation of SNNs were focused mainly on simple, feedforward, or oscillatory networks, but not much work has been done on optimising cortex-like recurrent SNNs. In this wor…
Real-time simulation of a large-scale biologically representative spiking neural network is presented, through the use of a heterogeneous parallelisation scheme and SpiNNaker neuromorphic hardware. A published cortical microcircuit model is used as a benchmark test case, representing approx. 1 square mm of early sensory cortex, containing 77k neurons and 0.3 billion synapses. This is the first true real-time simulation of this model, with 10 s of biological simulation time executed in 10 s wall-clock time. This surpasses best published efforts on HPC neural simulators (3x slowdown) and GPUs ru…
In the study of biological networks, one of the major challenges is to understand the relationships between network structure and dynamics. In this paper, we model in vitro cortical neuronal cultures as stochastic dynamical systems and apply a method that reconstructs directed networks from dynamics [Ching and Tam, Phys. Rev. E 95, 010301(R), 2017] to reveal directed effective connectivity, namely the directed links and synaptic weights, of the neuronal cultures from voltage measurements recorded by a multielectrode array. The effective connectivity so obtained reproduces several features of c…
Randomly-connected networks of integrate-and-fire (IF) neurons are known to display asynchronous irregular (AI) activity states, which resemble the discharge activity recorded in the cerebral cortex of awake animals. However, it is not clear whether such activity states are specific to simple IF models, or if they also exist in networks where neurons are endowed with complex intrinsic properties similar to electrophysiological measurements. Here, we investigate the occurrence of AI states in networks of nonlinear IF neurons, such as the adaptive exponential IF (Brette-Gerstner-Izhikevich) mode…
Several studies with brain signals suggested that bottom-up and top-down influences are exerted through distinct frequency bands among visual cortical areas. It has been recently shown that theta and gamma rhythms subserve feedforward, whereas the feedback influence is dominated by the alpha-beta rhythm in primates. A few theoretical models for reproducing these effects have been proposed so far. Here we show that a simple but biophysically plausible two-network motif composed of spiking-neuron models and chemical synapses can exhibit feedforward and feedback influences through distinct freque…
We demonstrate deterministic extensive chaos in the dynamics of large sparse networks of theta neurons in the balanced state. The analysis is based on numerically exact calculations of the full spectrum of Lyapunov exponents, the entropy production rate and the attractor dimension. Extensive chaos is found in inhibitory networks and becomes more intense when an excitatory population is included. We find a strikingly high rate of entropy production that would limit information representation in cortical spike patterns to the immediate stimulus response.
Fast cortical rhythms with stochastic and intermittent neural discharges have been observed in electric recordings of brain activity. Recently, Brunel et al. developed a framework to describe this kind of fast sparse synchronization in both random and globally-coupled networks of suprathreshold spiking neurons. However, in a real cortical circuit, synaptic connections are known to have complex topology which is neither regular nor random. Hence, in order to extend the works of Brunel et al. to realistic neural networks, we study the effect of network architecture on these fast sparsely synchro…
Broadband spontaneous macroscopic neural oscillations are rhythmic cortical firing which were extensively examined during the last century, however, their possible origination is still controversial. In this work we show how macroscopic oscillations emerge in solely excitatory random networks and without topological constraints. We experimentally and theoretically show that these oscillations stem from the counterintuitive underlying mechanism - the intrinsic stochastic neuronal response failures. These neuronal response failures, which are characterized by short-term memory, lead to cooperati…
Single trial analyses of ensemble activity in alert animals demonstrate that cortical circuits dynamics evolve through temporal sequences of metastable states. Metastability has been studied for its potential role in sensory coding, memory and decision-making. Yet, very little is known about the network mechanisms responsible for its genesis. It is often assumed that the onset of state sequences is triggered by an external stimulus. Here we show that state sequences can be observed also in the absence of overt sensory stimulation. Analysis of multielectrode recordings from the gustatory cortex…
Spike synchrony, which occurs in various cortical areas in response to specific perception, action and memory tasks, has sparked a long-standing debate on the nature of temporal organization in cortex. One prominent view is that this type of synchrony facilitates the binding or grouping of separate stimulus components. We argue instead for a more general function: A measure of the prior probability of incoming stimuli, implemented by long-range, horizontal, intra-cortical connections. We show that networks of this kind -- pulse-coupled excitatory spiking networks in a noisy environment -- can …
A rigorous understanding of brain dynamics and function requires a conceptual bridge between multiple levels of organization, including neural spiking and network-level population activity. Mounting evidence suggests that neural networks of cerebral cortex operate at criticality. How operating near this network state impacts the variability of neuronal spiking is largely unknown. Here we show in a computational model that two prevalent features of cortical single-neuron activity, irregular spiking and the decline of response variability at stimulus onset, are both emergent properties of a recu…
Spontaneous neuronal activity is a ubiquitous feature of cortex. Its spatiotemporal organization reflects past input and modulates future network output. Here we study whether a particular type of spontaneous activity is generated by a network that is optimized for input processing. Neuronal avalanches are a type of spontaneous activity observed in superficial cortical layers in vitro and in vivo with statistical properties expected from a network in a 'critical state'. Theory predicts that the critical state and, therefore, neuronal avalanches are optimal for input processing, but until now, …
Laboratory of Biological Modeling, NIDDK/NIH（美国国立糖尿病、消化系统疾病和肾病研究所（NIDDK）/美国国立卫生研究院（NIH）生物建模实验室）
Spiking activity of neurons engaged in learning and performing a task show complex spatiotemporal dynamics. While the output of recurrent network models can learn to perform various tasks, the possible range of recurrent dynamics that emerge after learning remains unknown. Here we show that modifying the recurrent connectivity with a recursive least squares algorithm provides sufficient flexibility for synaptic and spiking rate dynamics of spiking networks to produce a wide range of spatiotemporal activity. We apply the training method to learn arbitrary firing patterns, stabilize irregular sp…
During wakefulness and deep sleep brain states, cortical neural networks show a different behavior, with the second characterized by transients of high network activity. To investigate their impact on neuronal behavior, we apply a pairwise Ising model analysis by inferring the maximum entropy model that reproduces single and pairwise moments of the neuron's spiking activity. In this work we first review the inference algorithm introduced in Ferrari,Phys. Rev. E (2016). We then succeed in applying the algorithm to infer the model from a large ensemble of neurons recorded by multi-electrode arra…
The background activity of a cortical neural network is modeled by a homogeneous integrate-and-fire network with unreliable inhibitory synapses. Numerical and analytical calculations show that the network relaxes into a stationary state of high attention. The majority of the neurons has a membrane potential just below the threshold; as a consequence the network can react immediately - on the time scale of synaptic transmission- on external pulses. The neurons fire with a low rate and with a broad distribution of interspike intervals. Firing events of the total network are correlated over short…
A neural correlate of parametric working memory is a stimulus specific rise in neuron firing rate that persists long after the stimulus is removed. Network models with local excitation and broad inhibition support persistent neural activity, linking network architecture and parametric working memory. Cortical neurons receive noisy input fluctuations which causes persistent activity to diffusively wander about the network, degrading memory over time. We explore how cortical architecture that supports parametric working memory affects the diffusion of persistent neural activity. Studying both a …
The seemingly stochastic transient dynamics of neocortical circuits observed in vivo have been hypothesized to represent a signature of ongoing stochastic inference. In vitro neurons, on the other hand, exhibit a highly deterministic response to various types of stimulation. We show that an ensemble of deterministic leaky integrate-and-fire neurons embedded in a spiking noisy environment can attain the correct firing statistics in order to sample from a well-defined target distribution. We provide an analytical derivation of the activation function on the single cell level; for recurrent netwo…
Collective rhythmic dynamics from neurons is vital for cognitive functions such as memory formation but how neurons self-organize to produce such activity is not well understood. Attractor-based models have been successfully implemented as a theoretical framework for memory storage in networks of neurons. Activity-dependent modification of synaptic transmission is thought to be the physiological basis of learning and memory. The goal of this study is to demonstrate that using a pharmacological perturbation on in vitro networks of hippocampal neurons that has been shown to increase synaptic str…
The spontaneous activity pattern of cortical neurons in dissociated culture is characterized by burst firing that is highly synchronized among a wide population of cells. The degree of synchrony, however, is excessively higher than that in cortical tissues. Here, we employed polydimethylsiloxane (PDMS) elastomers to establish a novel system for culturing neurons on a scaffold with an elastic modulus resembling brain tissue, and investigated the effect of the scaffold's elasticity on network activity patterns in cultured rat cortical neurons. Using whole-cell patch clamp to assess the scaffold …
The synaptic connectivity of cortical networks features an overrepresentation of certain wiring motifs compared to simple random-network models. This structure is shaped, in part, by synaptic plasticity that promotes or suppresses connections between neurons depending on their spiking activity. Frequently, theoretical studies focus on how feedforward inputs drive plasticity to create this network structure. We study the complementary scenario of self-organized structure in a recurrent network, with spike timing-dependent plasticity driven by spontaneous dynamics. We develop a self-consistent t…
Excitation-inhibition (E-I) balance is ubiquitously observed in the cortex. Recent studies suggest an intriguing link between balance on fast timescales, tight balance, and efficient information coding with spikes. We further this connection by taking a principled approach to optimal balanced networks of excitatory (E) and inhibitory (I) neurons. By deriving E-I spiking neural networks from greedy spike-based optimizations of constrained minimax objectives, we show that tight balance arises from correcting for deviations from the minimax optima. We predict specific neuron firing rates in the n…
