Neuronal circuits internally regulate electrical signaling via a host of homeostatic mechanisms. Two prominent mechanisms, synaptic scaling and structural plasticity, are believed to maintain average activity within an operating range by modifying the strength and spatial extent of network connectivity using negative feedback. However, both mechanisms operate on relatively slow timescales and thus face fundamental limits due to delays. We show that these mechanisms fulfill complementary roles in maintaining stability in a large network. In particular, even relatively, slow growth dynamics improves performance significantly beyond synaptic scaling alone.
Excitatory synaptic connections in the adult neocortex consist of multiple synaptic contacts, almost exclusively formed on dendritic spines. Changes of dendritic spine shape and volume, a correlate of synaptic strength, can be tracked in vivo for weeks. Here, we present a combined model of spike-timing dependent dendritic spine plasticity and turnover that explains the steady state multi-contact configuration of synapses in adult neocortical networks. In this model, many presynaptic neurons compete to make strong synaptic connections onto postsynaptic neurons, while the synaptic contacts compr…
In long-term potentiation (LTP), one of the most studied types of neural plasticity, synaptic strength is persistently increased in response to stimulation. Although a number of different proteins have been implicated in the sub-cellular molecular processes underlying induction and maintenance of LTP, the precise mechanisms remain unknown. A particular challenge is to demonstrate that a proposed molecular mechanism can provide the level of stability needed to maintain memories for months or longer, in spite of the fact that many of the participating molecules have much shorter life spans. Here…
How can neural networks learn to efficiently represent complex and high-dimensional inputs via local plasticity mechanisms? Classical models of representation learning assume that input weights are learned via pairwise Hebbian-like plasticity. Here, we show that pairwise Hebbian-like plasticity only works under unrealistic requirements on neural dynamics and input statistics. To overcome these limitations, we derive from first principles a learning scheme based on voltage-dependent synaptic plasticity rules. Here, inhibition learns to locally balance excitatory input in individual dendritic co…
The main contribution to the local field potential (LFP) is thought to stem from synaptic input to neurons and the ensuing subthreshold dendritic processing. The role of active dendritic conductances in shaping the LFP has received little attention, even though such ion channels are known to affect the subthreshold neuron dynamics. Here we used a modeling approach to investigate the effects of subthreshold dendritic conductances on the LFP. Using a biophysically detailed, experimentally constrained model of a cortical pyramidal neuron, we identified conditions under which subthreshold active c…
In this paper we present a biologically detailed mathematical model of tripartite synapses, where astrocytes modulate short-term synaptic plasticity. The model consists of a pre-synaptic bouton, a post-synaptic dendritic spine-head, a synaptic cleft and a peri-synaptic astrocyte controlling Ca2+ dynamics inside the synaptic bouton. This in turn controls glutamate release dynamics in the cleft. As a consequence of this, glutamate concentration in the cleft has been modeled, in which glutamate reuptake by astrocytes has also been incorporated. Finally, dendritic spine-head dynamics has been mode…
Institute of Neuroinformatics（神经信息学研究所）；University of Zürich（苏黎世大学）；ETH Zürich（苏黎世联邦理工学院）；University of Bern（伯尔尼大学）；University of Bristol（布里斯托大学）；Mila（米拉研究所）；Université de Montréal（蒙特利尔大学）
Deep learning has seen remarkable developments over the last years, many of them inspired by neuroscience. However, the main learning mechanism behind these advances - error backpropagation - appears to be at odds with neurobiology. Here, we introduce a multilayer neuronal network model with simplified dendritic compartments in which error-driven synaptic plasticity adapts the network towards a global desired output. In contrast to previous work our model does not require separate phases and synaptic learning is driven by local dendritic prediction errors continuously in time. Such errors orig…
NSPDI-SNN: An efficient lightweight SNN based on nonlinear synaptic pruning and dendritic integration
Clinical Hospital of Chengdu Brain Science Institute, MOE Key Lab for NeuroInformation, China-Cuba Belt and Road Joint Laboratory on Neurotechnology and Brain-Apparatus Communication, School of Life Science and Technology, University of Electronic Science and Technology of China（成都脑科学研究所临床医院，教育部神经信息重点实验室，中老一带一路神经技术与脑-装置通信联合实验室，生命科学与技术学院，电子科技大学）；Academy of Medical Engineering and Translational Medicine, Tianjin University（医学工程与转化医学学院，天津大学）；School of Artificial Intelligence, Chongqing University of Education（人工智能学院，重庆教育学院）；Research Unit of NeuroInformation (2019RU035)（神经信息研究单位（2019RU035））
Department of Physiology University of Bern（伯尔尼大学生理学系）；Montreal Institute for Learning Algorithms Université de Montréal（蒙特利尔大学蒙特利尔学习算法研究所）
Animal behaviour depends on learning to associate sensory stimuli with the desired motor command. Understanding how the brain orchestrates the necessary synaptic modifications across different brain areas has remained a longstanding puzzle. Here, we introduce a multi-area neuronal network model in which synaptic plasticity continuously adapts the network towards a global desired output. In this model synaptic learning is driven by a local dendritic prediction error that arises from a failure to predict the top-down input given the bottom-up activities. Such errors occur at apical dendrites of …
Centre for Theoretical Neuroscience, University of Waterloo（滑铁卢大学理论神经科学中心）
Nonlinear interactions in the dendritic tree play a key role in neural computation. Nevertheless, modeling frameworks aimed at the construction of large-scale, functional spiking neural networks, such as the Neural Engineering Framework, tend to assume a linear superposition of post-synaptic currents. In this paper, we present a series of extensions to the Neural Engineering Framework that facilitate the construction of networks incorporating Dale's principle and nonlinear conductance-based synapses. We apply these extensions to a two-compartment LIF neuron that can be seen as a simple model o…
This paper presents a spike-based model which employs neurons with functionally distinct dendritic compartments for classifying high dimensional binary patterns. The synaptic inputs arriving on each dendritic subunit are nonlinearly processed before being linearly integrated at the soma, giving the neuron a capacity to perform a large number of input-output mappings. The model utilizes sparse synaptic connectivity; where each synapse takes a binary value. The optimal connection pattern of a neuron is learned by using a simple hardware-friendly, margin enhancing learning algorithm inspired by t…
Synaptic integration is a prominent aspect of neuronal information processing. The detailed mechanisms that modulate synaptic inputs determine the computational properties of any given neuron. We study a simple model for the summation of excitatory inputs from synapses and illustrate its use by characterizing some functional properties of postsynaptic neurons. In this regard, we study the response of postsynaptic neurons as defined by the model to two well known noise driven processes: stochastic and coherence resonance. The model requires a small number of parameters and is especially useful …
How neurons process their inputs crucially determines the dynamics of biological and artificial neural networks. In such neural and neural-like systems, synaptic input is typically considered to be merely transmitted linearly or sublinearly by the dendritic compartments. Yet, single-neuron experiments report pronounced supralinear dendritic summation of sufficiently synchronous and spatially close-by inputs. Here, we provide a statistical physics approach to study the impact of such non-additive dendritic processing on single neuron responses and the performance of associative memory tasks in …
Physiological experiments have highlighted how the dendrites of biological neurons can nonlinearly process distributed synaptic inputs. This is in stark contrast to units in artificial neural networks that are generally linear apart from an output nonlinearity. If dendritic trees can be nonlinear, biological neurons may have far more computational power than their artificial counterparts. Here we use a simple model where the dendrite is implemented as a sequence of thresholded linear units. We find that such dendrites can readily solve machine learning problems, such as MNIST or CIFAR-10, and …
Neuron is the most important building block in our brain, and information processing in individual neuron involves the transformation of input synaptic spike trains into an appropriate output spike train. Hardware implementation of neuron by individual ionic/electronic coupled device is of great importance for enhancing our understanding of the brain and solving sensory processing and complex recognition tasks. Here, we provide a proof-of-principle artificial neuron with multiple presynaptic inputs and one modulatory terminal based on a proton-coupled oxide-based electric-double-layer transist…
In many normative theories of synaptic plasticity, weight updates implicitly depend on the chosen parametrization of the weights. This problem relates, for example, to neuronal morphology: synapses which are functionally equivalent in terms of their impact on somatic firing can differ substantially in spine size due to their different positions along the dendritic tree. Classical theories based on Euclidean gradient descent can easily lead to inconsistencies due to such parametrization dependence. The issues are solved in the framework of Riemannian geometry, in which we propose that plasticit…
The studies described in this dissertation have attempted to address the cellular mechanisms of information storage by the brain. My work has focused primarily on the postsynaptic events that occur during and after induction of long-term potentiation (LTP) and long-term depression (LTD) at the Schaffer collateral-CA1 synapse of the hippocampus. I have explored various aspects of the role of postsynaptic Ca$^{2+}$ in the induction of LTP and LTD using both extracellular and whole-cell recording techniques, as well as fluorescence imaging of Ca$^{2+}$. I have also examined the possible role of m…
We present an effective model for timing-dependent synaptic plasticity (STDP) in terms of two interacting traces, corresponding to the fraction of activated NMDA receptors and the Ca2+ concentration in the dendritic spine of the postsynaptic neuron. This model intends to bridge the worlds of existing simplistic phenomenological rules and highly detailed models, constituting thus a practical tool for the study of the interplay between neural activity and synaptic plasticity in extended spiking neural networks. For isolated pairs of pre- and postsynaptic spikes the standard pairwise STDP rule is…
Circuits using superconducting single-photon detectors and Josephson junctions to perform signal reception, synaptic weighting, and integration are investigated. The circuits convert photon-detection events into flux quanta, the number of which is determined by the synaptic weight. The current from many synaptic connections is inductively coupled to a superconducting loop that implements the neuronal threshold operation. Designs are presented for synapses and neurons that perform integration as well as detect coincidence events for temporal coding. Both excitatory and inhibitory connections ar…
Dendritic spines, the carriers of long-term memory, occupy a small fraction of cortical space, and yet they are the major consumers of brain metabolic energy. What fraction of this energy goes for synaptic plasticity, correlated with learning and memory? It is estimated here based on neurophysiological and proteomic data for rat brain that, depending on the level of protein phosphorylation, the energy cost of synaptic plasticity constitutes a small fraction of the energy used for fast excitatory synaptic transmission, typically $4.0-11.2 \%$. Next, this study analyzes a metabolic cost of a new…
Membrane receptors for neuromodulators (NM) are highly regulated in their distribution and efficacy - a phenomenon which influences the individual cell's response to central signals of NM release. Even though NM receptor regulation is implicated in the pharmacological action of many drugs, and is also known to be influenced by various environmental factors, its functional consequences and modes of action are not well understood. In this paper we summarize relevant experimental evidence on NM receptor regulation (specifically dopamine D1 and D2 receptors) in order to explore its significance fo…
Replicating the computational functionalities and performances of the brain remains one of the biggest challenges for the future of information and communication technologies. Such an ambitious goal requires research efforts from the architecture level to the basic device level (i.e., investigating the opportunities offered by emerging nanotechnologies to build such systems). Nanodevices, or, more precisely, memory or memristive devices, have been proposed for the implementation of synaptic functions, offering the required features and integration in a single component. In this paper, we demon…
Neurons are spatially extended structures that receive and process inputs on their dendrites. It is generally accepted that neuronal computations arise from the active integration of synaptic inputs along a dendrite between the input location and the location of spike generation in the axon initial segment. However, many application such as simulations of brain networks, use point-neurons --neurons without a morphological component-- as computational units to keep the conceptual complexity and computational costs low. Inevitably, these applications thus omit a fundamental property of neuronal …
The role of gain neuromodulation in layer-5 pyramidal neurons
Neural Circuits Laboratory, Biosciences Institute, Faculty of Medical Sciences, Newcastle University（神经回路实验室、生物科学学院、医学科学学院、新castle大学）；Brain and Mind Center, The University of Sydney（脑与心智中心、悉尼大学）；Center for Complex Systems, The University of Sydney（复杂系统中心、悉尼大学）；IT:U Interdisciplinary Transformation University Austria（奥地利 interdisciplinary transformation university Austria）；International Research Center for Neurointelligence, The University of Tokyo（神经智能国际研究中心、东京大学）；Department of Anatomy, University of Quebec in Trois-Rivieres（解剖学系、魁北克三河大学）；Theoretical Sciences Visiting Program (TSVP), Okinawa Institute of Science and Technology Graduate University（理论科学访问计划（TSVP）、冲绳科学技术大学院大学）
To learn useful dynamics on long time scales, neurons must use plasticity rules that account for long-term, circuit-wide effects of synaptic changes. In other words, neural circuits must solve a credit assignment problem to appropriately assign responsibility for global network behavior to individual circuit components. Furthermore, biological constraints demand that plasticity rules are spatially and temporally local; that is, synaptic changes can depend only on variables accessible to the pre- and postsynaptic neurons. While artificial intelligence offers a computational solution for credit …
When Branch-Local Shunting Helps: A Gain-Load-Alignment Principle for Dendritic E/I Networks
Kempner Institute for the Study of Natural and Artificial Intelligence at Harvard University（哈佛大学生物与人工智能肯珀研究所）；Department of Neurobiology, Howard Hughes Medical Institute, Harvard Medical School（哈佛医学院神经生物学系、霍华德·休斯医学研究所）
Université de Montréal（蒙特利尔大学）；Montreal Institute for Learning Algorithms（蒙特利尔学习算法研究所）；University of Bern（伯尔尼大学）；Canadian Institute for Advanced Research（加拿大高等研究院）
We consider deep multi-layered generative models such as Boltzmann machines or Hopfield nets in which computation (which implements inference) is both recurrent and stochastic, but where the recurrence is not to model sequential structure, only to perform computation. We find conditions under which a simple feedforward computation is a very good initialization for inference, after the input units are clamped to observed values. It means that after the feedforward initialization, the recurrent network is very close to a fixed point of the network dynamics, where the energy gradient is 0. The ma…
Synaptic, dendritic and single-cell kinetics generate significant time delays that shape the dynamics of large networks of spiking neurons. Previous work has shown that such effective delays can be taken into account with a rate model through the addition of an explicit, fixed delay [Roxin et al. PRL 238103 (2005)]. Here we extend this work to account for arbitrary symmetric patterns of synaptic connectivity and generic nonlinear transfer functions. Specifically, we conduct a weakly nonlinear analysis of the dynamical states arising via primary instabilities of the asynchronous state. In this …
When neural circuits learn to perform a task, it is often the case that there are many sets of synaptic connections that are consistent with the task. However, only a small number of possible solutions are robust to noise in the input and are capable of generalizing their performance of the task to new inputs. Finding such good solutions is an important goal of learning systems in general and neuronal circuits in particular. For systems operating with static inputs and outputs, a well known approach to the problem is the large margin methods such as Support Vector Machines (SVM). By maximizing…
Sequence learning, prediction and replay have been proposed to constitute the universal computations performed by the neocortex. The Hierarchical Temporal Memory (HTM) algorithm realizes these forms of computation. It learns sequences in an unsupervised and continuous manner using local learning rules, permits a context specific prediction of future sequence elements, and generates mismatch signals in case the predictions are not met. While the HTM algorithm accounts for a number of biological features such as topographic receptive fields, nonlinear dendritic processing, and sparse connectivit…
An Operating Principle of the Cerebral Cortex, and a Cellular Mechanism for Attentional Trial-and-Error Pattern Learning and Useful Classification Extraction
Degeneracy in epilepsy: Multiple Routes to Hyperexcitable Brain Circuits and their Repair
Frankfurt Institute for Advanced Studies（法兰克福高等研究院）；Cellular Neurophysiology Laboratory, Molecular Biophysics Unit, Indian Institute of Science（印度科学学院分子生物物理学单元细胞神经生理学实验室）；ICAR3R – Interdisciplinary Centre for 3Rs in Animal Research, Faculty of Medicine, Justus Liebig University Giessen（吉森尤斯图斯·李比希大学医学院动物研究3R跨学科中心）；Institute of Clinical Neuroanatomy, Neuroscience Center, Goethe University（歌德大学神经科学中心临床神经解剖学研究所）
