The response of neurons is highly sensitive to the stimulus. The stimulus can be associated with a direct injection in vitro experimentation (e.g., time dependent and independent inputs); or post-synaptic potentials resulting from the interaction of many neurons. A typical incoming stimulus resembles a noise which in principle can be described as a random variable. In computational neuroscience, the noise has been extensively studied for different setups. In this study, we investigate the effect of noisy inputs in a minimal network of two identical leaky integrate-and-fire (LIF) neurons interacting with finite pulses. In particular, we consider a Gaussian white noise as a standard function for stochastic modelling of neurons, while taking into account the pulse width as an elementary component for the signal transmission. By exploring the role of noise and finite pulses, the two neurons show a synchronous spiking behaviour characterized by fluctuations in the interspike intervals. Above some critical values the synchronous regime collapses onto asynchronous dynamics. The abrupt change in such dynamics is accompanied by a hysteresis, i.e., the coexistence of synchronous and asynchronous firing behaviour.
In this paper the objectivity of the description of charged particles transport in electrical circuits, in biological neurons and biological neuron-networks is discussed. It is shown that the use of Riemann-Liouville or Caputo fractional order derivatives leads to the loose of the description objectivity.
Physiological experiments have highlighted how the dendrites of biological neurons can nonlinearly process distributed synaptic inputs. This is in stark contrast to units in artificial neural networks that are generally linear apart from an output nonlinearity. If dendritic trees can be nonlinear, biological neurons may have far more computational power than their artificial counterparts. Here we use a simple model where the dendrite is implemented as a sequence of thresholded linear units. We find that such dendrites can readily solve machine learning problems, such as MNIST or CIFAR-10, and …
This article highlights specific features of biological neurons and their dendritic trees, whose adoption may help advance artificial neural networks used in various machine learning applications. Advancements could take the form of increased computational capabilities and/or reduced power consumption. Proposed features include dendritic anatomy, dendritic nonlinearities, and compartmentalized plasticity rules, all of which shape learning and information processing in biological networks. We discuss the computational benefits provided by these features in biological neurons and suggest ways to…
Multistability, the coexistence of multiple attractors in a dynamical system, is explored in bursting nerve cells. A modeling study is performed to show that a large class of bursting systems, as defined by a shared topology when represented as dynamical systems, is inherently suited to support multistability. We derive the bifurcation structure and parametric trends leading to multistability in these systems. Evidence for the existence of multirhythmic behavior in neurons of the aquatic mollusc Aplysia californica that is consistent with our proposed mechanism is presented. Although these exp…
In a neuron with hyperpolarization activated current ($I_h$), the correct input frequency leads to an enhancement of the output response. This behavior is known as resonance and is well described by the neuronal impedance. In a simple neuron model we derive equations for the neuron's resonance and we link its frequency and existence with the biophysical properties of $I_h$. For a small voltage change, the component of the ratio of current change to voltage change ($dI/dV$) due to the voltage-dependent conductance change ($dg/dV$) is known as derivative conductance ($G_h^{Der}$). We show that b…
Classification of biological neuron types and networks poses challenges to the full understanding of the brain's organisation and functioning. In this paper, we develop a novel objective classification model of biological neuronal types and networks based on the communication metrics of neurons. This presents advantages against the existing approaches since the mutual information or the delay between neurons obtained from spike trains are more abundant data compare to conventional morphological data. We firstly designed two open-access supporting computational platforms of various neuronal cir…
This work introduces the Koha model, a new theory that aims to explain two unresolved phenomena within biological neural networks: How information is processed and stored within neural circuits, and how neurons learn to become pattern detectors. In the Koha model, the dendritic spines of a neuron serve as computational units that scan for precise spike patterns in their synaptic inputs. The model proposes the existence of a temporal code within each dendritic spine, which is used for the dampening or amplification of signals, depending on the temporal information of incoming spike trains. Comp…
There are several indications that brain is organized not on a basis of individual unreliable neurons, but on a micro-circuital scale providing Lego blocks employed to create complex architectures. At such an intermediate scale, the firing activity in the microcircuits is governed by collective effects emerging by the background noise soliciting spontaneous firing, the degree of mutual connections between the neurons, and the topology of the connections. We compare spontaneous firing activity of small populations of neurons adhering to an engineered scaffold with simulations of biologically pl…
Many biological neuronal networks exhibit highly variable spiking activity. Balanced networks offer a parsimonious model of this variability. In balanced networks, strong excitatory synaptic inputs are canceled by strong inhibitory inputs on average and spiking activity is driven by transient breaks in this balance. Most previous studies of balanced networks assume a homogeneous or distance-dependent connectivity structure, but connectivity in biological cortical networks is more intricate. We use a heterogeneous mean-field theory of balanced networks to show that heterogeneous in-degrees can …
There has been a strong push recently to examine biological scale simulations of neuromorphic algorithms to achieve stronger inference capabilities. This paper presents a set of piecewise linear spiking neuron models, which can reproduce different behaviors, similar to the biological neuron, both for a single neuron as well as a network of neurons. The proposed models are investigated, in terms of digital implementation feasibility and costs, targeting large scale hardware implementation. Hardware synthesis and physical implementations on FPGA show that the proposed models can produce precise …
This paper has been withdrawn. Its main conclusions have been published in "On dynamics of integrate-and-fi re neural networks with conductance based synapses", arXiv:0709.4370 and http://www.frontiersin.org/computational_neuroscience/10.3389/neuro.10/002.2008/abstract
In this paper, we propose a shot noise-based leaky integrated and firing neuron model and provide a detailed analysis of the performance of this model compared to the traditional diffusion approximated model. In theoretical neuroscience, there are three general neuron models in the field: Compartmental neuron model is a conductance-based model, in which it views the biological neurons as a large circuit. The problem of this model comes from its structural complexity and the number of its free parameters; Leaky integrated and firing model is a more flexible model due to the special design calle…
A method of data assimilation (DA) is employed to estimate electrophysiological parameters of neurons simultaneously with their synaptic connectivity in a small model biological network. The DA procedure is cast as an optimization, with a cost function consisting of both a measurement error and a model error term. An iterative reweighting of these terms permits a systematic method to identify the lowest minimum, within a local region of state space, on the surface of a non-convex cost function. In the model, two sets of parameter values are associated with two particular functional modes of ne…
With the goal of understanding the intricate behavior and dynamics of collections of neurons, we present superconducting circuits containing Josephson junctions that model biologically realistic neurons. These "Josephson junction neurons" reproduce many characteristic behaviors of biological neurons such as action potentials, refractory periods, and firing thresholds. They can be coupled together in ways that mimic electrical and chemical synapses. Using existing fabrication technologies, large interconnected networks of Josephson junction neurons would operate fully in parallel. They would be…
In this work we study the detection of weak stimuli by spiking neurons in the presence of certain level of noisy background neural activity. Our study has focused in the realistic assumption that the synapses in the network present activity-dependent processes, such as short-term synaptic depression and facilitation. Employing mean-field techniques as well as numerical simulations, we found that there are two possible noise levels which optimize signal transmission. This new finding is in contrast with the classical theory of stochastic resonance which is able to predict only one optimal level…
NOBLE -- Neural Operator with Biologically-informed Latent Embeddings to Capture Experimental Variability in Biological Neuron Models
ETH Zürich（苏黎世联邦理工学院）；California Institute of Technology（加州理工学院）；University of Alberta（阿尔伯塔大学）；Alberta Machine Intelligence Institute (Amii)（阿尔伯塔人工智能研究所）；Cedars-Sinai Medical Center（Cedars-Sinai 医疗中心）；Archimedes AI, Athena Research Center（Archimedes AI，Athena 研究中心）
Biological Neurons Compete with Deep Reinforcement Learning in Sample Efficiency in a Simulated Gameworld
Turner Institute for Brain and Mental Health（特纳脑与心理健康研究所）；Monash University（莫纳什大学）；Cortical Labs（皮层实验室）；IITB-Monash Research Academy（印度理工学院孟买分校-莫纳什大学研究院）；Wellcome Centre for Human Neuroimaging（惠康人类神经影像中心）；University College London（伦敦大学学院）
First return maps of interspike intervals for biological neurons that generate repetitive bursts of impulses can display stereotyped structures (neuronal signatures). Such structures have been linked to the possibility of multicoding and multifunctionality in neural networks that produce and control rhythmical motor patterns. In some cases, isolating the neurons from their synaptic network revealsirregular, complex signatures that have been regarded as evidence of intrinsic, chaotic behavior. We show that incorporation of dynamical noise into minimal neuron models of square-wave bursting (eith…
Computations on the dendritic trees of neurons have important constraints. Voltage dependent conductances in dendrites are not similar to arbitrary direct-current generation, they are the basis for dendritic nonlinearities and they do not allow converting positive currents into negative currents. While it has been speculated that the dendritic tree of a neuron can be seen as a multi-layer neural network and it has been shown that such an architecture could be computationally strong, we do not know if that computational strength is preserved under these biological constraints. Here we simulate …
Artificial neural networks (ANNs) have been extensively used for the description of problems arising from biological systems and for constructing neuromorphic computing models. The third generation of ANNs, namely, spiking neural networks (SNNs), inspired by biological neurons enable a more realistic mimicry of the human brain. A large class of the problems from these domains is characterized by the necessity to deal with the combination of neurons, spikes and synapses via integrate-and-fire neuron models. Motivated by important applications of the integrate-and-fire of neurons in neuromorphic…
The mammalian brain could contain dense and sparse network connectivity structures, including both excitatory and inhibitory neurons, but is without any clearly defined output layer. The neurons have time constants, which mean that the integrated network structure has state memory. The network structure contains complex mutual interactions between the neurons under different conditions, which depend on the internal state of the network. The internal state can be defined as the distribution of activity across all individual neurons across the network. Therefore, the state of a neuron/network be…
Inhibitory neurons play a crucial role in maintaining persistent neuronal activity. Although connected extensively through electrical synapses (gap-junctions), these neurons also exhibit interactions through chemical synapses in certain regions of the brain. When the coupling is sufficiently strong, the effects of these two synaptic modalities combine in a nonlinear way. Hence, in this work, we focus on the strong inhibition regime and identify the parametric conditions that result in the emergence of self-sustained oscillations in systems of coupled excitable neurons, in the presence of a bri…
Harvard University（哈佛大学）；MIT-IBM Watson AI Lab（麻省理工-IBM沃森人工智能实验室）；IBM Research（IBM研究院）
While deep neural networks take loose inspiration from neuroscience, it is an open question how seriously to take the analogies between artificial deep networks and biological neuronal systems. Interestingly, recent work has shown that deep convolutional neural networks (CNNs) trained on large-scale image recognition tasks can serve as strikingly good models for predicting the responses of neurons in visual cortex to visual stimuli, suggesting that analogies between artificial and biological neural networks may be more than superficial. However, while CNNs capture key properties of the average…
Neuro-inspired models and systems have great potential for applications in unconventional computing. Often, the mechanisms of biological neurons are modeled or mimicked in simulated or physical systems in an attempt to harness some of the computational power of the brain. However, the biological mechanisms at play in neural systems are complicated and challenging to capture and engineer; thus, it can be simpler to turn to a data-driven approach to transfer features of neural behavior to artificial substrates. In the present study, we used an evolutionary algorithm (EA) to produce spiking neura…
Biological cortical networks are potentially fully recurrent networks without any distinct output layer, where recognition may instead rely on the distribution of activity across its neurons. Because such biological networks can have rich dynamics, they are well-designed to cope with dynamical interactions of the types that occur in nature, while traditional machine learning networks may struggle to make sense of such data. Here we connected a simple model neuronal network (based on the 'linear summation neuron model' featuring biologically realistic dynamics (LSM), consisting of 10 of excitat…
Bio-integrated neuromorphic systems promise for new protocols to record and regulate the signaling of biological systems. Making such artificial neural circuits successful requires minimal circuit complexity and ion-based operating mechanisms similar to that of biology. However, simple leaky integrate-and-fire model neurons, commonly realized in either silicon or organic semiconductor neuromorphic systems, can emulate only a few neural features. More functional neuron models, based on traditional complex Si-based complementary-metal-oxide-semiconductor (CMOS) or negative differential resistanc…
An electronic circuit device, inspired on the FitzHugh-Nagumo model of neuronal excitability, was constructed and shown to operate with characteristics compatible with those of biological sensory neurons. The nonlinear dynamical model of the electronics quantitatively reproduces the experimental observations on the circuit, including the Hopf bifurcation at the onset of tonic spiking. Moreover, we have implemented an analog noise generator as a source to study the variability of the spike trains. When the circuit is in the excitable regime, coherence resonance is observed. At sufficiently low …
Carbon nanotubes are often seen as the only alternative technology to silicon transistors. While they are the most likely short-term one, other longer-term alternatives should be studied as well. While contemplating biological neurons as an alternative component may seem preposterous at first sight, significant recent progress in CMOS-neuron interface suggests this direction may not be unrealistic; moreover, biological neurons are known to self-assemble into very large networks capable of complex information processing tasks, something that has yet to be achieved with other emerging technologi…
Newcastle University（纽卡斯尔大学）；Okinawa Institute of Science and Technology（冲绳科学技术大学院大学）；University of York（约克大学）
Local field potentials (LFPs) sampled with extracellular electrodes are frequently used as a measure of population neuronal activity. However, relating such measurements to underlying neuronal behaviour and connectivity is non-trivial. To help study this link, we developed the Virtual Electrode Recording Tool for EXtracellular potentials (VERTEX). We first identified a reduced neuron model that retained the spatial and frequency filtering characteristics of extracellular potentials from neocortical neurons. We then developed VERTEX as an easy-to-use Matlab tool for simulating LFPs from large p…
In recent years, several studies have provided insight on the functioning of the brain which consists of neurons and form networks via interconnection among them by synapses. Neural networks are formed by interconnected systems of neurons, and are of two types, namely, the Artificial Neural Network (ANNs) and Biological Neural Network (interconnected nerve cells). The ANNs are computationally influenced by human neurons and are used in modelling neural systems. The reasoning foundations of ANNs have been useful in anomaly detection, in areas of medicine such as instant physician, electronic no…
Cortical pyramidal neurons receive inputs from multiple distinct neural populations and integrate these inputs in separate dendritic compartments. We explore the possibility that cortical microcircuits implement Canonical Correlation Analysis (CCA), an unsupervised learning method that projects the inputs onto a common subspace so as to maximize the correlations between the projections. To this end, we seek a multi-channel CCA algorithm that can be implemented in a biologically plausible neural network. For biological plausibility, we require that the network operates in the online setting and…
One conjecture in both deep learning and classical connectionist viewpoint is that the biological brain implements certain kinds of deep networks as its back-end. However, to our knowledge, a detailed correspondence has not yet been set up, which is important if we want to bridge between neuroscience and machine learning. Recent researches emphasized the biological plausibility of Linear-Nonlinear-Poisson (LNP) neuron model. We show that with neurally plausible settings, the whole network is capable of representing any Boltzmann machine and performing a semi-stochastic Bayesian inference algor…
What fascinates us about animal behavior is its richness and complexity, but understanding behavior and its neural basis requires a simpler description. Traditionally, simplification has been imposed by training animals to engage in a limited set of behaviors, by hand scoring behaviors into discrete classes, or by limiting the sensory experience of the organism. An alternative is to ask whether we can search through the dynamics of natural behaviors to find explicit evidence that these behaviors are simpler than they might have been. We review two mathematical approaches to simplification, dim…
In network models of spiking neurons, the joint impact of network structure and synaptic parameters on activity propagation is still an open problem. Here we use an information-theoretical approach to investigate activity propagation in spiking networks with hierarchical modular topology. We observe that optimized pairwise information propagation emerges due to the increase of either (i) the global synaptic strength parameter or (ii) the number of modules in the network, while the network size remains constant. At the population level, information propagation of activity among adjacent modules…
This paper presents a multi-point stimulation of a Hebbian neural network with investigation of the interplay between the stimulus waves through the neurons of the network. Equilibrium of the resulting memory is achieved for recall of specific memory data at a rate faster than single point stimulus. The interplay of the intersecting stimuli appears to parallel the clarification process of recall in biological systems.
Field closure, ice neurons, and when a dendrite is a motif
McGovern Institute for Brain Research, Massachusetts Institute of Technology（麻省理工学院麦戈文脑研究所）；The NSF AI Institute for Artificial Intelligence and Fundamental Interactions (IAIFI)（美国国家科学基金会人工智能基础相互作用研究所）
Living organisms must actively maintain themselves in order to continue existing. Autopoiesis is a key concept in the study of living organisms, where the boundaries of the organism is not static by dynamically regulated by the system itself. To study the autonomous regulation of self-boundary, we focus on neural homeodynamic responses to environmental changes using both biological and artificial neural networks. Previous studies showed that embodied cultured neural networks and spiking neural networks with spike-timing dependent plasticity (STDP) learn an action as they avoid stimulation from…
An artificial neural network (ANN) based on fundamental principles of physics can simulate the operation of neurobiological reality of membrane potential as well as the properly defined order parameter. This ANN operates in conditions of criticality and simulates the behavior of an excitatory biological neuron, especially the relaxation phase where the critical fluctuations of biological neuron appear. These critical fluctuations can not be explained by the Hodgkin-Huxley (H-H) model. A proposal for the origin of these fluctuations is being discussed.
Fault-Tolerant Neural Networks from Biological Error Correction Codes
Center for Theoretical Physics, Massachusetts Institute of Technology（麻省理工学院理论物理中心）；Department of Physics, Massachusetts Institute of Technology（麻省理工学院物理系）；Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology（麻省理工学院电气工程与计算机科学系）
The Expressive Leaky Memory Neuron: an Efficient and Expressive Phenomenological Neuron Model Can Solve Long-Horizon Tasks
University of Tübingen（蒂宾根大学）；Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）；Mila, Quebec AI Institute（米拉魁北克人工智能研究所）；Max Planck Institute for Biological Cybernetics（马克斯·普朗克生物控制论研究所）
Spontaneous formation of clusters of synchronized spiking in a structureless ensemble of equal stochastically perturbed excitable neurons with delayed coupling is demonstrated for the first time. The effect is a consequence of a subtle interplay between interaction delays, noise and the excitable character of a single neuron. Dependence of the cluster properties on the time-lag, noise intensity and the synaptic strength is investigated.
Spike-timing dependent plasticity (STDP) which observed in the brain has proven to be important in biological learning. On the other hand, artificial neural networks use a different way to learn, such as Back-Propagation or Contrastive Hebbian Learning. In this work, we propose a new framework called mstdp that learn almost the same way biological learning use, it only uses STDP rules for supervised and unsupervised learning and don' t need a global loss or other supervise information. The framework works like an auto-encoder by making each input neuron also an output neuron. It can make predi…
Using an exactly solvable cortical model of a neuronal network, we show that, by increasing the intensity of shot noise (flow of random spikes bombarding neurons), the network undergoes first- and second-order non-equilibrium phase transitions. We study the nature of the transitions, bursts and avalanches of neuronal activity. Saddle-node and supercritical Hopf bifurcations are the mechanisms of emergence of sustained network oscillations. We show that the network stimulated by shot noise behaves similar to the Morris-Lecar model of a biological neuron stimulated by an applied current.
We investigate the dynamics of two models of biological networks with purely suppressive interactions between the units; species interacting via niche competition and neurons via inhibitory synaptic coupling. In both of these cases, power-law scaling of the density of states with probability arises without any fine-tuning of the model parameters. These results argue against the increasingly popular notion that non-equilibrium living systems operate at special critical points, driven by there by evolution so as to enable adaptive processing of input data.
The timing of individual neuronal spikes is essential for biological brains to make fast responses to sensory stimuli. However, conventional artificial neural networks lack the intrinsic temporal coding ability present in biological networks. We propose a spiking neural network model that encodes information in the relative timing of individual neuron spikes. In classification tasks, the output of the network is indicated by the first neuron to spike in the output layer. This temporal coding scheme allows the supervised training of the network with backpropagation, using locally exact derivati…
