How neurons integrate the myriad synaptic inputs scattered across their dendrites is a fundamental question in neuroscience. Multiple neurophysiological experiments have shown that dendritic non-linearities can have a strong influence on synaptic input integration. These non-linearities have motivated mathematical descriptions of single neuron as a two-layer computational units, which have been shown to increase substantially the computational abilities of neurons, compared to linear dendritic integration. However, current analytical studies are restricted to neurons with unconstrained synaptic weights and unplausible dendritic non-linearities. Here, we introduce a two-layer model with sign-constrained synaptic weights and a biologically plausible form of dendritic non-linearity, and investigate its properties using both statistical physics methods and numerical simulations. We find that the dendritic non-linearity enhances both the number of possible learned input-output associations and the learning velocity. We characterize how capacity and learning speed depend on the implemented non-linearity and the levels of dendritic and somatic inhibition. We calculate analytically the distribution of synaptic weights in networks close to maximal capacity, and find that a large fraction of zero-weight ('silent' or 'potential') synapses naturally emerge in neurons with sign-constrained synapses, as a consequence of non-linear dendritic integration. Non-linearly induced sparsity comes with a second central advantage for neuronal information processing, i.e. input and synaptic noise robustness. We test our model on standard real-world benchmark datasets and observe empirically that the non-linearity provides an enhancement in generalization performance, showing that it enables to capture more complex input/output relations.
Neuron is the most important building block in our brain, and information processing in individual neuron involves the transformation of input synaptic spike trains into an appropriate output spike train. Hardware implementation of neuron by individual ionic/electronic coupled device is of great importance for enhancing our understanding of the brain and solving sensory processing and complex recognition tasks. Here, we provide a proof-of-principle artificial neuron with multiple presynaptic inputs and one modulatory terminal based on a proton-coupled oxide-based electric-double-layer transist…
Physiological experiments have highlighted how the dendrites of biological neurons can nonlinearly process distributed synaptic inputs. This is in stark contrast to units in artificial neural networks that are generally linear apart from an output nonlinearity. If dendritic trees can be nonlinear, biological neurons may have far more computational power than their artificial counterparts. Here we use a simple model where the dendrite is implemented as a sequence of thresholded linear units. We find that such dendrites can readily solve machine learning problems, such as MNIST or CIFAR-10, and …
Neurons are thought of as the building blocks of excitable brain tissue. However, at the single neuron level, the neuronal membrane, the dendritic arbor and the axonal projections can also be considered an extended active medium. Active dendritic branchlets enable the propagation of dendritic spikes, whose computational functions, despite several proposals, remain an open question. Here we propose a concrete function to the active channels in large dendritic trees. By using a probabilistic cellular automaton approach, we model the input-output response of large active dendritic arbors subjecte…
Top-down feedback in cortex is critical for guiding sensory processing, which has prominently been formalized in the theory of hierarchical predictive coding (hPC). However, experimental evidence for error units, which are central to the theory, is inconclusive, and it remains unclear how hPC can be implemented with spiking neurons. To address this, we connect hPC to existing work on efficient coding in balanced networks with lateral inhibition, and predictive computation at apical dendrites. Together, this work points to an efficient implementation of hPC with spiking neurons, where predictio…
How neurons process their inputs crucially determines the dynamics of biological and artificial neural networks. In such neural and neural-like systems, synaptic input is typically considered to be merely transmitted linearly or sublinearly by the dendritic compartments. Yet, single-neuron experiments report pronounced supralinear dendritic summation of sufficiently synchronous and spatially close-by inputs. Here, we provide a statistical physics approach to study the impact of such non-additive dendritic processing on single neuron responses and the performance of associative memory tasks in …
How can neural networks learn to efficiently represent complex and high-dimensional inputs via local plasticity mechanisms? Classical models of representation learning assume that input weights are learned via pairwise Hebbian-like plasticity. Here, we show that pairwise Hebbian-like plasticity only works under unrealistic requirements on neural dynamics and input statistics. To overcome these limitations, we derive from first principles a learning scheme based on voltage-dependent synaptic plasticity rules. Here, inhibition learns to locally balance excitatory input in individual dendritic co…
Computations on the dendritic trees of neurons have important constraints. Voltage dependent conductances in dendrites are not similar to arbitrary direct-current generation, they are the basis for dendritic nonlinearities and they do not allow converting positive currents into negative currents. While it has been speculated that the dendritic tree of a neuron can be seen as a multi-layer neural network and it has been shown that such an architecture could be computationally strong, we do not know if that computational strength is preserved under these biological constraints. Here we simulate …
The nonlinear properties of the dendrites in prepositus hypoglossi neurons are involved in maintenance of eye position. The biophysical properties of these neurons are essential for the operation of the vestibular neural integrator that converts a head velocity signal to one that controls eye position. A novel method named QSA (quadratic sinusoidal analysis) for voltage clamped neurons was used to quantify nonlinear responses that are dominated by dendrites. The voltage clamp currents were measured at harmonic and interactive frequencies using specific stimulation frequencies, which act as fre…
Cortical pyramidal neurons have a complex dendritic anatomy, whose function is an active research field. In particular, the segregation between its soma and the apical dendritic tree is believed to play an active role in processing feed-forward sensory information and top-down or feedback signals. In this work, we use a simple two-compartment model accounting for the nonlinear interactions between basal and apical input streams and show that standard unsupervised Hebbian learning rules in the basal compartment allow the neuron to align the feed-forward basal input with the top-down target sign…
Since the first experimental evidences of active conductances in dendrites, most neurons have been shown to exhibit dendritic excitability through the expression of a variety of voltage-gated ion channels. However, despite experimental and theoretical efforts undertaken in the last decades, the role of this excitability for some kind of dendritic computation has remained elusive. Here we show that, owing to very general properties of excitable media, the average output of a model of active dendritic trees is a highly non-linear function of their afferent rate, attaining extremely large dynamic…
Recent experiments revealed that a certain class of inhibitory neurons in the cerebral cortex make synapses not onto cell bodies but at distal parts of dendrites of the target neurons, mediating highly nonlinear dendritic inhibition. We propose a novel form of competitive neural network model that realizes such dendritic inhibition. Contrary to the conventional lateral inhibition in neural networks, our dendritic inhibition models don't always show winner-take-all behaviors; instead, they converge to "I don't know" states when unknown input patterns are presented. We derive reduced two-dimensi…
This article highlights specific features of biological neurons and their dendritic trees, whose adoption may help advance artificial neural networks used in various machine learning applications. Advancements could take the form of increased computational capabilities and/or reduced power consumption. Proposed features include dendritic anatomy, dendritic nonlinearities, and compartmentalized plasticity rules, all of which shape learning and information processing in biological networks. We discuss the computational benefits provided by these features in biological neurons and suggest ways to…
Excitatory synaptic connections in the adult neocortex consist of multiple synaptic contacts, almost exclusively formed on dendritic spines. Changes of dendritic spine shape and volume, a correlate of synaptic strength, can be tracked in vivo for weeks. Here, we present a combined model of spike-timing dependent dendritic spine plasticity and turnover that explains the steady state multi-contact configuration of synapses in adult neocortical networks. In this model, many presynaptic neurons compete to make strong synaptic connections onto postsynaptic neurons, while the synaptic contacts compr…
The brain as a neuronal system has very complex structure with large diversity of neuronal types. The most basic complexity is seen from the structure of neuronal morphology, which usually has a complex tree-like structure with dendritic spines distributed in branches. For simulating a large-scale network with spiking neurons, the simple point neuron, such as integrate-and-fire neuron, is often used. However, recent experimental evidence suggests that the computational ability of a single neuron is largely enhanced by its morphological structure, in particular, by various types of dendritic dy…
The main contribution to the local field potential (LFP) is thought to stem from synaptic input to neurons and the ensuing subthreshold dendritic processing. The role of active dendritic conductances in shaping the LFP has received little attention, even though such ion channels are known to affect the subthreshold neuron dynamics. Here we used a modeling approach to investigate the effects of subthreshold dendritic conductances on the LFP. Using a biophysically detailed, experimentally constrained model of a cortical pyramidal neuron, we identified conditions under which subthreshold active c…
We present a biophysical approach for the coupling of neural network activity as resulting from proper dipole currents of cortical pyramidal neurons to the electric field in extracellular fluid. Starting from a reduced threecompartment model of a single pyramidal neuron, we derive an observation model for dendritic dipole currents in extracellular space and thereby for the dendritic field potential that contributes to the local field potential of a neural population. This work aligns and satisfies the widespread dipole assumption that is motivated by the "open-field" configuration of the dendr…
Neurons are spatially extended structures that receive and process inputs on their dendrites. It is generally accepted that neuronal computations arise from the active integration of synaptic inputs along a dendrite between the input location and the location of spike generation in the axon initial segment. However, many application such as simulations of brain networks, use point-neurons --neurons without a morphological component-- as computational units to keep the conceptual complexity and computational costs low. Inevitably, these applications thus omit a fundamental property of neuronal …
Neuronal morphology is an essential element for brain activity and function. We take advantage of current availability of brain-wide neuron digital reconstructions of the Pyramidal cells from a mouse brain, and analyze several emergent features of brain-wide neuronal morphology. We observe that axonal trees are self-affine while dendritic trees are self-similar. We also show that tree size appear to be random, independent of the number of dendrites within single neurons. Moreover, we consider inhomogeneous branching model which stochastically generates rooted 3-Cayley trees for the brain-wide …
Conceptual and mathematical models of neurons have lagged behind empirical understanding for decades. Here we extend previous work in modeling biological systems with fully scale-independent quantum information-theoretic tools to develop a uniform, scalable representation of synapses, dendritic and axonal processes, neurons, and local networks of neurons. In this representation, hierarchies of quantum reference frames act as hierarchical active-inference systems. The resulting model enables specific predictions of correlations between synaptic activity, dendritic remodeling, and trophic reward…
Physics-constrained inference of somatic dynamics from dendritic recordings with sparse somatic supervision in weakly coupled two-compartment neuron model
Université Le Havre Normandie（勒阿弗尔诺曼底大学）；The Hudson School of Mathematics（哈德森数学学院）
Centre for Theoretical Neuroscience, University of Waterloo（滑铁卢大学理论神经科学中心）
Nonlinear interactions in the dendritic tree play a key role in neural computation. Nevertheless, modeling frameworks aimed at the construction of large-scale, functional spiking neural networks, such as the Neural Engineering Framework, tend to assume a linear superposition of post-synaptic currents. In this paper, we present a series of extensions to the Neural Engineering Framework that facilitate the construction of networks incorporating Dale's principle and nonlinear conductance-based synapses. We apply these extensions to a two-compartment LIF neuron that can be seen as a simple model o…
NSPDI-SNN: An efficient lightweight SNN based on nonlinear synaptic pruning and dendritic integration
Clinical Hospital of Chengdu Brain Science Institute, MOE Key Lab for NeuroInformation, China-Cuba Belt and Road Joint Laboratory on Neurotechnology and Brain-Apparatus Communication, School of Life Science and Technology, University of Electronic Science and Technology of China（成都脑科学研究所临床医院，教育部神经信息重点实验室，中老一带一路神经技术与脑-装置通信联合实验室，生命科学与技术学院，电子科技大学）；Academy of Medical Engineering and Translational Medicine, Tianjin University（医学工程与转化医学学院，天津大学）；School of Artificial Intelligence, Chongqing University of Education（人工智能学院，重庆教育学院）；Research Unit of NeuroInformation (2019RU035)（神经信息研究单位（2019RU035））
Department of Physiology University of Bern（伯尔尼大学生理学系）；Montreal Institute for Learning Algorithms Université de Montréal（蒙特利尔大学蒙特利尔学习算法研究所）
Animal behaviour depends on learning to associate sensory stimuli with the desired motor command. Understanding how the brain orchestrates the necessary synaptic modifications across different brain areas has remained a longstanding puzzle. Here, we introduce a multi-area neuronal network model in which synaptic plasticity continuously adapts the network towards a global desired output. In this model synaptic learning is driven by a local dendritic prediction error that arises from a failure to predict the top-down input given the bottom-up activities. Such errors occur at apical dendrites of …
This paper describes the Synapto-dendritic Kernel Adapting Neuron (SKAN), a simple spiking neuron model that performs statistical inference and unsupervised learning of spatiotemporal spike patterns. SKAN is the first proposed neuron model to investigate the effects of dynamic synapto-dendritic kernels and demonstrate their computational power even at the single neuron scale. The rule-set defining the neuron is simple there are no complex mathematical operations such as normalization, exponentiation or even multiplication. The functionalities of SKAN emerge from the real-time interaction of si…
How extracellular electric fields, as generated endogenously or through transcranial brain stimulation, affect the dynamics of large neuronal populations is of great interest but not well understood. To study the collective dynamics of large populations single-compartment (point) model neurons have been proven very successful. These models, however, lack the dendritic morphology to biophysically account for the effects of electric fields, and for changes in synaptic integration due to morphology alone. Here we (i) characterize the response of a canonical spatial (ball-and-stick) model neuron t…
Institute of Neuroinformatics（神经信息学研究所）；University of Zürich（苏黎世大学）；ETH Zürich（苏黎世联邦理工学院）；University of Bern（伯尔尼大学）；University of Bristol（布里斯托大学）；Mila（米拉研究所）；Université de Montréal（蒙特利尔大学）
Deep learning has seen remarkable developments over the last years, many of them inspired by neuroscience. However, the main learning mechanism behind these advances - error backpropagation - appears to be at odds with neurobiology. Here, we introduce a multilayer neuronal network model with simplified dendritic compartments in which error-driven synaptic plasticity adapts the network towards a global desired output. In contrast to previous work our model does not require separate phases and synaptic learning is driven by local dendritic prediction errors continuously in time. Such errors orig…
This paper presents a spike-based model which employs neurons with functionally distinct dendritic compartments for classifying high dimensional binary patterns. The synaptic inputs arriving on each dendritic subunit are nonlinearly processed before being linearly integrated at the soma, giving the neuron a capacity to perform a large number of input-output mappings. The model utilizes sparse synaptic connectivity; where each synapse takes a binary value. The optimal connection pattern of a neuron is learned by using a simple hardware-friendly, margin enhancing learning algorithm inspired by t…
Neuronal circuits internally regulate electrical signaling via a host of homeostatic mechanisms. Two prominent mechanisms, synaptic scaling and structural plasticity, are believed to maintain average activity within an operating range by modifying the strength and spatial extent of network connectivity using negative feedback. However, both mechanisms operate on relatively slow timescales and thus face fundamental limits due to delays. We show that these mechanisms fulfill complementary roles in maintaining stability in a large network. In particular, even relatively, slow growth dynamics impr…
The role of gain neuromodulation in layer-5 pyramidal neurons
Neural Circuits Laboratory, Biosciences Institute, Faculty of Medical Sciences, Newcastle University（神经回路实验室、生物科学学院、医学科学学院、新castle大学）；Brain and Mind Center, The University of Sydney（脑与心智中心、悉尼大学）；Center for Complex Systems, The University of Sydney（复杂系统中心、悉尼大学）；IT:U Interdisciplinary Transformation University Austria（奥地利 interdisciplinary transformation university Austria）；International Research Center for Neurointelligence, The University of Tokyo（神经智能国际研究中心、东京大学）；Department of Anatomy, University of Quebec in Trois-Rivieres（解剖学系、魁北克三河大学）；Theoretical Sciences Visiting Program (TSVP), Okinawa Institute of Science and Technology Graduate University（理论科学访问计划（TSVP）、冲绳科学技术大学院大学）
In many normative theories of synaptic plasticity, weight updates implicitly depend on the chosen parametrization of the weights. This problem relates, for example, to neuronal morphology: synapses which are functionally equivalent in terms of their impact on somatic firing can differ substantially in spine size due to their different positions along the dendritic tree. Classical theories based on Euclidean gradient descent can easily lead to inconsistencies due to such parametrization dependence. The issues are solved in the framework of Riemannian geometry, in which we propose that plasticit…
The electric activities of cortical pyramidal neurons are supported by structurally stable, morphologically complex axo-dendritic trees. Anatomical differences between axons and dendrites in regard to their length or caliber reflect the underlying functional specializations, for input or output of neural information, respectively. For a proper assessment of the computational capacity of pyramidal neurons, we have analyzed an extensive dataset of three-dimensional digital reconstructions from the NeuroMorpho.Org database, and quantified basic dendritic or axonal morphometric measures in differe…
Synaptic integration is a prominent aspect of neuronal information processing. The detailed mechanisms that modulate synaptic inputs determine the computational properties of any given neuron. We study a simple model for the summation of excitatory inputs from synapses and illustrate its use by characterizing some functional properties of postsynaptic neurons. In this regard, we study the response of postsynaptic neurons as defined by the model to two well known noise driven processes: stochastic and coherence resonance. The model requires a small number of parameters and is especially useful …
Technical University of Crete（克里特理工大学）；Foundation for Research and Technology-Hellas（研究与科技基金会-赫尔拉斯）
Current deep learning architectures show remarkable performance when trained in large-scale, controlled datasets. However, the predictive ability of these architectures significantly decreases when learning new classes incrementally. This is due to their inclination to forget the knowledge acquired from previously seen data, a phenomenon termed catastrophic-forgetting. On the other hand, Self-Organizing Maps (SOMs) can model the input space utilizing constrained k-means and thus maintain past knowledge. Here, we propose a novel algorithm inspired by biological neurons, termed Dendritic-Self-Or…
Field closure, ice neurons, and when a dendrite is a motif
McGovern Institute for Brain Research, Massachusetts Institute of Technology（麻省理工学院麦戈文脑研究所）；The NSF AI Institute for Artificial Intelligence and Fundamental Interactions (IAIFI)（美国国家科学基金会人工智能基础相互作用研究所）
Pairs of neurons in brain networks often share much of the input they receive from other neurons. Due to essential non-linearities of neuronal dynamics, the consequences for the correlation of the output spike trains are not well understood in the strongly correlated regime. Here we consider two leaky integrate-and-fire neurons with correlated white noise input. We analyze this scenario using a novel non-perturbative approach. Hence our treatment covers both weakly and strongly correlated dynamics, generalizing previous results based on linear response theory.
Learning and memory may rely on the ability of neuronal circuits to reorganize by dendritic spine remodeling. We have looked for geometrical parameters of cortical circuits, which maximize information storage capacity associated with this mechanism. In particular, we calculated optimal volume fractions of various neuropil components. The optimal axonal and dendritic volume fractions are not significantly different from anatomical measurements in the mouse and rat neocortex, and the rat hippocampus. This has led us to propose that the maximization of information storage capacity associated with…
The studies described in this dissertation have attempted to address the cellular mechanisms of information storage by the brain. My work has focused primarily on the postsynaptic events that occur during and after induction of long-term potentiation (LTP) and long-term depression (LTD) at the Schaffer collateral-CA1 synapse of the hippocampus. I have explored various aspects of the role of postsynaptic Ca$^{2+}$ in the induction of LTP and LTD using both extracellular and whole-cell recording techniques, as well as fluorescence imaging of Ca$^{2+}$. I have also examined the possible role of m…
Circuits using superconducting single-photon detectors and Josephson junctions to perform signal reception, synaptic weighting, and integration are investigated. The circuits convert photon-detection events into flux quanta, the number of which is determined by the synaptic weight. The current from many synaptic connections is inductively coupled to a superconducting loop that implements the neuronal threshold operation. Designs are presented for synapses and neurons that perform integration as well as detect coincidence events for temporal coding. Both excitatory and inhibitory connections ar…
We have added a simplified neuromorphic model of Spike Time Dependent Plasticity (STDP) to the Synapto-dendritic Kernel Adapting Neuron (SKAN). The resulting neuron model is the first to show synaptic encoding of afferent signal to noise ratio in addition to the unsupervised learning of spatio temporal spike patterns. The neuron model is particularly suitable for implementation in digital neuromorphic hardware as it does not use any complex mathematical operations and uses a novel approach to achieve synaptic homeostasis. The neurons noise compensation properties are characterized and tested o…
Neocortical pyramidal neurons have many dendrites, and such dendrites are capable of, in isolation of one-another, generating a neuronal spike. It is also now understood that there is a large amount of dendritic growth during the first years of a humans life, arguably a period of prodigious learning. These observations inspire the construction of a local, stochastic algorithm based on an earlier stochastic, Hebbian developmental theory. Here we investigate the neuro-computational advantages and limits on this novel algorithm that combines dendritogenesis with supervised adaptive synaptogenesis…
The Expressive Leaky Memory Neuron: an Efficient and Expressive Phenomenological Neuron Model Can Solve Long-Horizon Tasks
University of Tübingen（蒂宾根大学）；Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）；Mila, Quebec AI Institute（米拉魁北克人工智能研究所）；Max Planck Institute for Biological Cybernetics（马克斯·普朗克生物控制论研究所）
Neuron is the most important building block in our brain, and information processing in individual neuron involves the transformation of input synaptic spike trains into an appropriate output spike train. Hardware implementation of neuron by individual ionic/electronic hybrid device is of great significance for enhancing our understanding of the brain and solving sensory processing and complex recognition tasks. Here, we provide a proof-of-principle artificial neuron based on a proton conducting graphene oxide (GO) coupled oxide-based electric-double-layer (EDL) transistor with multiple drivin…
Serotonergic neurons of the dorsal raphe nuclei, with their extensive innervation of nearly the whole brain have important modulatory effects on many cognitive and physiological processes. They play important roles in clinical depression and other psychiatric disorders. In order to quantify the effects of serotonergic transmission on target cells it is desirable to construct computational models and to this end these it is necessary to have details of the biophysical and spike properties of the serotonergic neurons. Here several basic properties are reviewed with data from several studies sinc…
When neural circuits learn to perform a task, it is often the case that there are many sets of synaptic connections that are consistent with the task. However, only a small number of possible solutions are robust to noise in the input and are capable of generalizing their performance of the task to new inputs. Finding such good solutions is an important goal of learning systems in general and neuronal circuits in particular. For systems operating with static inputs and outputs, a well known approach to the problem is the large margin methods such as Support Vector Machines (SVM). By maximizing…
