Modeling brain dynamics to better understand and control complex behaviors underlying various cognitive brain functions are of interests to engineers, mathematicians, and physicists from the last several decades. With a motivation of developing computationally efficient models of brain dynamics to use in designing control-theoretic neurostimulation strategies, we have developed a novel data-driven approach in a long short-term memory (LSTM) neural network architecture to predict the temporal dynamics of complex systems over an extended long time-horizon in future. In contrast to recent LSTM-based dynamical modeling approaches that make use of multi-layer perceptrons or linear combination layers as output layers, our architecture uses a single fully connected output layer and reversed-order sequence-to-sequence mapping to improve short time-horizon prediction accuracy and to make multi-timestep predictions of dynamical behaviors. We demonstrate the efficacy of our approach in reconstructing the regular spiking to bursting dynamics exhibited by an experimentally-validated 9-dimensional Hodgkin-Huxley model of hippocampal CA1 pyramidal neurons. Through simulations, we show that our LSTM neural network can predict the multi-time scale temporal dynamics underlying various spiking patterns with reasonable accuracy. Moreover, our results show that the predictions improve with increasing predictive time-horizon in the multi-timestep deep LSTM neural network.
We establish a general linear response relation for spiking neuronal networks, based on chains with unbounded memory. This relation allows us to predict the influence of a weak amplitude time-dependent external stimuli on spatio-temporal spike correlations, from the spontaneous statistics (without stimulus) in a general context where the memory in spike dynamics can extend arbitrarily far in the past. Using this approach, we show how linear response is explicitly related to neuronal dynamics with an example, the gIF model, introduced by M. Rudolph and A. Destexhe. This example illustrates the …
In this article we present a biologically inspired model of activation of memory items in a sequence. Our model produces two types of sequences, corresponding to two different types of cerebral functions: activation of regular or irregular sequences. The switch between the two types of activation occurs through the modulation of biological parameters, without altering the connectivity matrix. Some of the parameters included in our model are neuronal gain, strength of inhibition, synaptic depression and noise. We investigate how these parameters enable the existence of sequences and influence t…
Neuronal networks provide living organisms with the ability to process information. They are also characterized by abundant recurrent connections, which give rise to strong feedback that dictates their dynamics and endows them with fading (short-term) memory. The role of recurrence in long-term memory, on the other hand, is still unclear. Here we use the neuronal network of the roundworm C. elegans to show that recurrent architectures in living organisms can exhibit long-term memory without relying on specific hard-wired modules. A genetic algorithm reveals that the experimentally observed dyn…
Activity Regeneration from Silent States in Neuronal Networks with Transient Synaptic Memory
Pasargad Institute for Advanced Innovative Solutions (PIAIS)（帕萨加德先进创新解决方案研究所）；Zapata Briceño Institute for Human Intelligence Research（萨帕塔·布里塞尼奥人类智能研究所）；Institute for Advanced Studies in Basic Sciences (IASBS)（基础科学高等研究院）
The mechanism of memory localization in extended domains is described in the framework of the parametric dissipative quantum model of brain. The size of the domains and the capability in memorizing depend on the number of links the system is able to establish with the external world.
A number of cortical structures are reported to have elevated single unit firing rates sustained throughout the memory period of a working memory task. How the nervous system forms and maintains these memories is unknown but reverberating neuronal network activity is thought to be important. We studied the temporal structure of single unit (SU) activity and simultaneously recorded local field potential (LFP) activity from area LIP in the inferior parietal lobe of two awake macaques during a memory-saccade task. Using multitaper techniques for spectral analysis, which play an important role in …
A model of the columnar functional organization of neocortical association areas is studied. The neuronal network is composed of many Hebbian autoassociators, or modules, each of which interacts with a relatively small number of the others. Every module encodes and stores a number of elementary percepts, or features. Memory items, or patterns, are peculiar combinations of features sparsely distributed over the multi-modular network. Any feature stored in any module can be involved in several of the stored patterns; feature-sharing is in fact source of local ambiguities and, consequently, a pot…
We consider a simple Markovian class of the stochastic Wilson-Cowan type models of neuronal network dynamics, which incorporates stochastic delay caused by the existence of a refractory period of neurons. From the point of view of the dynamics of the individual elements, we are dealing with a network of non-Markovian stochastic two-state oscillators with memory which are coupled globally in a mean-field fashion. This interrelation of a higher-dimensional Markovian and lower-dimensional non-Markovian dynamics is discussed in its relevance to the general problem of the network dynamics of comple…
The brain is believed to operate in part by making predictions about sensory stimuli and encoding deviations from these predictions in the activity of "prediction error neurons." This principle defines the widely influential theory of predictive coding. The precise circuitry and plasticity mechanisms through which animals learn to compute and update their predictions are unknown. Homeostatic inhibitory synaptic plasticity is a promising mechanism for training neuronal networks to perform predictive coding. Homeostatic plasticity causes neurons to maintain a steady, baseline firing rate in resp…
The oculomotor delayed-response (ODR) task is a common experimental paradigm of working memory (WM) study, in which a monkey must fixate its gaze on the center of a screen and, following a brief cue that flashes on the screen, keep fixating for several more seconds before shifting its gaze to the location where the cue flashed. Consequently, in the delay period between the cue and the response the monkey must maintain a memory of cue location. Electrophysiological recordings from the prefrontal area of the cortex (PFC) revealed neurons that display selective persistent activity: their firing r…
The spiking activity of principal cells in mammalian hippocampus encodes an internalized neuronal representation of the ambient space---a cognitive map. Once learned, such a map enables the animal to navigate a given environment for a long period. However, the neuronal substrate that produces this map remains transient: the synaptic connections in the hippocampus and in the downstream neuronal networks never cease to form and to deteriorate at a rapid rate. How can the brain maintain a robust, reliable representation of space using a network that constantly changes its architecture? Here, we d…
The brain as an astonishingly remarkable device has been studied from various angles. It is now well known that neurons are the seat of all activities of the brain function. The dynamical properties pertaining to a single neuron and a collection of neurons may be widely different owing to the clustering properties of a group of neurons. As it can be clearly understood theory of complex physical systems has been more and more employed to study the behaviour of neurons and neuronal circuits. We here mainly discuss neural correlates of memory and cognitive functions utilizing graph theory and ide…
Understanding the basic operational logics of the nervous system is essential to advancing neuroscientific research. However, theoretical efforts to tackle this fundamental problem are lacking, despite the abundant empirical data about the brain that has been collected in the past few decades. To address this shortcoming, this document introduces a hypothetical framework for the functional nature of primitive neural networks. It analyzes the idea that the activity of neurons and synapses can symbolically reenact the dynamic changes in the world and thus enable an adaptive system of behavior. M…
In this correspondence information theoretical tools are used to investigate the statistical properties of modeled cochlear nucleus globular bushy cell spike trains. The firing patterns are obtained from a simulation software that generates sample spike trains from any auditory input. Here we analyze for the first time the responses of globular bushy cells to voiced and unvoiced speech sounds. Classical entropy estimates, such as the direct method, are improved upon by considering a time-varying and time-dependent entropy estimate. With this method we investigated the relationship between the …
Recording simultaneous activity of hundreds of neurons is now possible. Existing methods can model such population activity, but do not directly reveal the computations used by the brain. We present a fully unsupervised method that models neuronal activity and reveals the computational strategy. The method constructs a topological model of neuronal dynamics consisting of interconnected loops. Transitions between loops mark computationally-salient decisions. We accurately model activation of 100s of neurons in the primate cortex during a working memory task. Dynamics of a recurrent neural netwo…
Hippocampal cognitive map---a neuronal representation of the spatial environment---is broadly discussed in the computational neuroscience literature for decades. More recent studies point out that hippocampus plays a major role in producing yet another cognitive framework that incorporates not only spatial, but also nonspatial memories---the memory space. However, unlike cognitive maps, memory spaces have been barely studied from a theoretical perspective. Here we propose an approach for modeling hippocampal memory spaces as an epiphenomenon of neuronal spiking activity. First, we suggest that…
Increasing experimental evidence suggests that axonal action potential conduction velocity is a highly adaptive parameter in the adult central nervous system. Yet, the effects of this newfound plasticity on global brain dynamics is poorly understood. In this work, we analyzed oscillations in biologically plausible neuronal networks with different conduction velocity distributions. Changes of 1-2 (ms) in network mean signal transmission time resulted in substantial network oscillation frequency changes ranging in 0-120 (Hz). Our results suggest that changes in axonal conduction velocity may sig…
During sleep, the hippocampus recapitulates neuronal patterns corresponding to behavioral trajectories during previous experiences. This hippocampal replay supports the formation of long-term memories. Yet, whether replay originates within the hippocampal circuitry or is initiated by extrahippocampal inputs is unknown. Here, I review recent findings regarding the organization of neuronal activity upstream to the hippocampus, in the head-direction (HD) and grid cell networks. I argue that hippocampal activity is under the influence of primary spatial signals, which originate from subcortical st…
The neuronal paradigm of studying the brain has left us with limitations in both our understanding of how neurons process information to achieve biological intelligence and how such knowledge may be translated into artificial intelligence and its most brain-derived branch, neuromorphic computing. Overturning our fundamental assumptions of how the brain works, the recent exploration of astrocytes is revealing that these long-neglected brain cells dynamically regulate learning by interacting with neuronal activity at the synaptic level. Following recent experimental evidence, we designed an asso…
The Chinese University of Hong Kong（香港中文大学）；Southeast University（东南大学）；Cornell University（康奈尔大学）
Throughout history, the development of artificial intelligence, particularly artificial neural networks, has been open to and constantly inspired by the increasingly deepened understanding of the brain, such as the inspiration of neocognitron, which is the pioneering work of convolutional neural networks. Per the motives of the emerging field: NeuroAI, a great amount of neuroscience knowledge can help catalyze the next generation of AI by endowing a network with more powerful capabilities. As we know, the human brain has numerous morphologically and functionally different neurons, while artifi…
Broadband spontaneous macroscopic neural oscillations are rhythmic cortical firing which were extensively examined during the last century, however, their possible origination is still controversial. In this work we show how macroscopic oscillations emerge in solely excitatory random networks and without topological constraints. We experimentally and theoretically show that these oscillations stem from the counterintuitive underlying mechanism - the intrinsic stochastic neuronal response failures. These neuronal response failures, which are characterized by short-term memory, lead to cooperati…
For constructing neuronal network models computational neuroscientists have access to wide-ranging anatomical data that nevertheless tend to cover only a fraction of the parameters to be determined. Finding and interpreting the most relevant data, estimating missing values, and combining the data and estimates from various sources into a coherent whole is a daunting task. With this chapter we aim to provide guidance to modelers by describing the main types of anatomical data that may be useful for informing neuronal network models. We further discuss aspects of the underlying experimental tech…
The representation of the natural-density, heterogeneous connectivity of neuronal network models at relevant spatial scales remains a challenge for Computational Neuroscience and Neuromorphic Computing. In particular, the memory demands imposed by the vast number of synapses in brain-scale network simulations constitutes a major obstacle. Limiting the number resolution of synaptic weights appears to be a natural strategy to reduce memory and compute load. In this study, we investigate the effects of a limited synaptic-weight resolution on the dynamics of recurrent spiking neuronal networks res…
Neocortical pyramidal neurons have many dendrites, and such dendrites are capable of, in isolation of one-another, generating a neuronal spike. It is also now understood that there is a large amount of dendritic growth during the first years of a humans life, arguably a period of prodigious learning. These observations inspire the construction of a local, stochastic algorithm based on an earlier stochastic, Hebbian developmental theory. Here we investigate the neuro-computational advantages and limits on this novel algorithm that combines dendritogenesis with supervised adaptive synaptogenesis…
Starting from the morphological-functional assumption of the fractal brain, a mathematical model is given by activating brain non-differentiable dynamics through the determinism-nondeterminism inference of the responsible mechanisms. The postulation of a scale covariance principle in Schrodinger type representation of the brain geodesics implies the spectral functionality of the brain dynamics through mechanisms of tunelling, percolation etc., while in the hydrodynamical type representation, it implies their structural functionality through mechanisms of wave schock, solitons type etc. For ext…
The brain forms and stores distributed representations from sparse external input that compete for neuronal resources with already stored memory traces. It is unclear what dynamical properties of neural systems allow formation and subsequent consolidation of new, distributed memory representations under these conditions. Here we use analytical, computational, and experimental approaches to show that a dynamical regime near a phase-transition in neuronal network activity (i.e. criticality) may play an important role in this process. Our results reveal that near-critical dynamics are necessary t…
Brain research has been driven by enquiry for principles of brain structure organization and its control mechanisms. The neuronal wiring map of C. elegans, the only complete connectome available till date, presents an incredible opportunity to learn basic governing principles that drive structure and function of its neuronal architecture. Despite its apparently simple nervous system, C. elegans is known to possess complex functions. The neuronal architecture forms an important underlying framework which specifies phenotypic features associated to sensation, movement, conditioning and memory. I…
Lobachevsky State University of Nizhny Novgorod（罗巴切夫斯基国立下诺夫哥罗德大学）；Institute of Applied Physics of the Russian Academy of Sciences（俄罗斯科学院应用物理研究所）；Immanuel Kant Baltic Federal University（伊曼努尔·康德波罗的海联邦大学）；Sechenov First Moscow State Medical University (Sechenov University)（谢切诺夫第一莫斯科国立医科大学（谢切诺夫大学））；University College London（伦敦大学学院）；Innopolis University（因诺波利斯大学）
Modeling the neuronal processes underlying short-term working memory remains the focus of many theoretical studies in neuroscience. Here we propose a mathematical model of spiking neuron network (SNN) demonstrating how a piece of information can be maintained as a robust activity pattern for several seconds then completely disappear if no other stimuli come. Such short-term memory traces are preserved due to the activation of astrocytes accompanying the SNN. The astrocytes exhibit calcium transients at a time scale of seconds. These transients further modulate the efficiency of synaptic transm…
Synaptic connections in neuronal circuits are modulated by pre- and post-synaptic spiking activity. Heuristic models of this process of synaptic plasticity can provide excellent fits to results from in-vitro experiments in which pre- and post-synaptic spiking is varied in a controlled fashion. However, the plasticity rules inferred from fitting such data are inevitably unstable, in that given constant pre- and post-synaptic activity the synapse will either fully potentiate or depress. This instability can be held in check by adding additional mechanisms, such as homeostasis. Here we consider a…
Columbia University（哥伦比亚大学）；Edmond and Lily Safra Center for Brain Sciences（埃德蒙和莉莉·萨弗拉脑科学中心）；Racah Institute of Physics（拉卡物理研究所）；The Hebrew University of Jerusalem（耶路撒冷希伯来大学）；Center for Brain Science（脑科学中心）；Harvard University（哈佛大学）
Neurons and networks in the cerebral cortex must operate reliably despite multiple sources of noise. To evaluate the impact of both input and output noise, we determine the robustness of single-neuron stimulus selective responses, as well as the robustness of attractor states of networks of neurons performing memory tasks. We find that robustness to output noise requires synaptic connections to be in a balanced regime in which excitation and inhibition are strong and largely cancel each other. We evaluate the conditions required for this regime to exist and determine the properties of networks…
Computational modeling and simulation have become essential tools in the quest to better understand the brain's makeup and to decipher the causal interrelations of its components. The breadth of biochemical and biophysical processes and structures in the brain has led to the development of a large variety of model abstractions and specialized tools, often times requiring high performance computing resource for their timely execution. What has been missing so far was an in-depth analysis of the complexity of the computational kernels, hindering a systematic approach to identifying bottlenecks o…
Neurodegenerative diseases and traumatic brain injuries (TBI) are among the main causes of cognitive dysfunction in humans. Both manifestations exhibit the extensive presence of focal axonal swellings (FAS). FAS compromises the information encoded in spike trains, thus leading to potentially severe functional deficits. Complicating our understanding of the impact of FAS is our inability to access small scale injuries with non-invasive methods, the overall complexity of neuronal pathologies, and our limited knowledge of how networks process biological signals. Building on Hopfield's pioneering …
Alzheimer's disease (AD) is a common form of dementia observed in the elderly due to neurodegenerative disorder and dysfunction. This arises from alterations in synaptic functioning of neurons leading to cognitive impairment and memory loss. Recent experimental studies indicate that the amyloid beta (A beta) protein, in its dimer and oligomer forms, affects the synaptic activity of neurons in the early stages of AD. However, the precise mechanism underlying A beta induced synaptic depression is still not clearly understood. In this paper, we introduce an electrical model that provides a possib…
Firing Rate Models as Associative Memory: Excitatory-Inhibitory Balance for Robust Retrieval
Department of Information Engineering（信息工程系）；Università degli Studi di Padova（帕多瓦大学）；Center for Control, Dynamical Systems and Computation（控制、动态系统与计算中心）；University of California at Santa Barbara（加州大学圣巴巴拉分校）
Random matrix theory of sparse neuronal networks with heterogeneous timescales
Complex Systems, Department of Physics, Faculty of Science, Chulalongkorn University, Bangkok, Thailand；High Energy Physics Research Unit, Faculty of Science, Chulalongkorn University, Bangkok, Thailand；International Solvay Institutes, Brussels, Belgium；Mark Kac Center for Complex Systems Research, Jagiellonian University, Krak\'ow, Poland；Department of Mathematics, King's College London, London, UK
Simplified neuronal models capture the essence of the electrical activity of a generic neuron, besides being more interesting from the computational point of view when compared to higher dimensional models such as the Hodgkin-Huxley one. In this work, we propose a generalized resonate-and-fire model described by a generalized Langevin equation that takes into account memory effects and colored noise. We perform a comprehensive numerical analysis to study the dynamics and the point process statistics of the proposed model, highlighting interesting new features like: i) non-monotonic behavior (e…
Harvard University（哈佛大学）；MIT-IBM Watson AI Lab（麻省理工-IBM沃森人工智能实验室）；IBM Research（IBM研究院）
While deep neural networks take loose inspiration from neuroscience, it is an open question how seriously to take the analogies between artificial deep networks and biological neuronal systems. Interestingly, recent work has shown that deep convolutional neural networks (CNNs) trained on large-scale image recognition tasks can serve as strikingly good models for predicting the responses of neurons in visual cortex to visual stimuli, suggesting that analogies between artificial and biological neural networks may be more than superficial. However, while CNNs capture key properties of the average…
Neuronal electricality founded in murburn-thermodynamic principles: 2. Comparisons, evidenced explanations, and predictions
Amrita School of Artificial Intelligence, Coimbatore, Amrita Vishwa Vidyapeetham（阿姆利塔人工智能学院，阿姆利塔维迪亚佩塔姆大学）；Satyamjayatu: The Science & Ethics Foundation（真理至上：科学与伦理基金会）；University of Rostock（罗斯托克大学）；Wayne State University（韦恩州立大学）
Recently, researchers have found time cells in the hippocampus that appear to contain information about the timing of past events. Some researchers have argued that time cells are taking a Laplace transform of their input in order to reconstruct the past stimulus. We argue that stimulus prediction, not stimulus reconstruction or redundancy reduction, is in better agreement with observed responses of time cells. In the process, we introduce new analyses of nonlinear, continuous-time reservoirs that model these time cells.
Late long-term potentiation (L-LTP) appears essential for the formation of long-term memory, with memories at least partly encoded by patterns of strengthened synapses. How memories are preserved for months or years, despite molecular turnover, is not well understood. Ongoing recurrent neuronal activity, during memory recall or during sleep, has been hypothesized to preferentially potentiate strong synapses, preserving memories. This hypothesis has not been evaluated in the context of a mathematical model representing biochemical pathways important for L-LTP. I incorporated ongoing activity in…
Although recent neurophysiological experiments suggest that synchronous neural activity is involved in some perceptual and cognitive processes, the functional role of such coherent neuronal behavior is not well understood. As a first step in clarifying this role, we investigate how the temporal coherence of certain neuronal activity affects the activity pattern in a neural network. Using a simple network of leaky integrate-and-fire neurons, we study the effects of synchronized incoming spikes on the functioning of two mechanisms typically used in model neural systems, winner-take-all competiti…
Recurrent networks of dynamic elements frequently exhibit emergent collective oscillations, which can display substantial regularity even when the individual elements are considerably noisy. How noise-induced dynamics at the local level coexists with regular oscillations at the global level is still unclear. Here we show that a combination of stochastic recurrence-based initiation with deterministic refractoriness in an excitable network can reconcile these two features, leading to maximum collective coherence for an intermediate noise level. We report this behavior in the slow oscillation reg…
