Cognitive processes in the brain, like learning, formation of memory, recovery of memorized images, classification of objects have two features: First, there is no supervisor in the brain who controls these processes. Second there is a hugh number of neurons (10^{6} to 10^{10}) involved in those cognitive tasks. For this reason, the search of understanding cognitive processes uses models built from a large number of neurons, but very much simplified neurons. The so-called neural networks have been quite successful in describing certain aspects of brain functions, like the mechanism of associative memory or recently the prediction of epileptic seizures. At hand of the Kohonen network we discuss the treatment of information in the brain, in particular how the brain organizes such information without supervisor. Recently, networks of small-world and scale-free architecture came into focus. There is evidence indicating that the brain (cat cortex, macaque cortex, human brain) uses such connectivity architecture. Tasks like treatment of information, learning and classification take advantage of such scale-free and small-world connectivity and thus play a potentially important role in self-organization of the brain.
Networks of living neurons exhibit an avalanche mode of activity, experimentally found in organotypic cultures. Here we present a model based on self-organized criticality and taking into account brain plasticity, which is able to reproduce the spectrum of electroencephalograms (EEG). The model consists in an electrical network with threshold firing and activity-dependent synapse strenghts. The system exhibits an avalanche activity power law distributed. The analysis of the power spectra of the electrical signal reproduces very robustly the power law behaviour with the exponent 0.8, experiment…
Many complex systems display self-organized critical states characterized by 1/f frequency scaling of power spectra. Global variables such as the electroencephalogram, scale as 1/f, which could be the sign of self-organized critical states in neuronal activity. By analyzing simultaneous recordings of global and neuronal activities, we confirm the 1/f scaling of global variables for selected frequency bands, but show that neuronal activity is not consistent with critical states. We propose a model of 1/f scaling which does not rely on critical states, and which is testable experimentally.
The critical brain hypothesis receives increasing support from recent experimental results. It postulates that the brain is at a critical point between an ordered and a chaotic regime, sometimes referred to as the "edge of chaos." Another central observation of neuroscience is the principle of excitation-inhibition balance: Certain brain networks exhibit a remarkably constant ratio between excitation and inhibition. When this balance is perturbed, the network shifts away from the critical point, as may for example happen during epileptic seizures. However, it is as of yet unclear what mechanis…
The brain keeps its overall dynamics in a corridor of intermediate activity and it has been a long standing question what possible mechanism could achieve this task. Mechanisms from the field of statistical physics have long been suggesting that this homeostasis of brain activity could occur even without a central regulator, via self-organization on the level of neurons and their interactions, alone. Such physical mechanisms from the class of self-organized criticality exhibit characteristic dynamical signatures, similar to seismic activity related to earthquakes. Measurements of cortex rest a…
The liquid state machine (LSM) combines low training complexity and biological plausibility, which has made it an attractive machine learning framework for edge and neuromorphic computing paradigms. Originally proposed as a model of brain computation, the LSM tunes its internal weights without backpropagation of gradients, which results in lower performance compared to multi-layer neural networks. Recent findings in neuroscience suggest that astrocytes, a long-neglected non-neuronal brain cell, modulate synaptic plasticity and brain dynamics, tuning brain networks to the vicinity of the comput…
The free energy principle has been proposed as a unifying account of brain function. It is closely related, and in some cases subsumes, earlier unifying ideas such as Bayesian inference, predictive coding, and active learning. This article clarifies these connections, teasing apart distinctive and shared predictions.
Recent studies on the complex systems have shown that the synchronization of oscillators including neuronal ones is faster, stronger, and more efficient in the small-world networks than in the regular or the random networks, and many studies are based on the assumption that the brain may utilize the small-world and scale-free network structure. We show that the functional structures in the brain are self-organized to both the small-world and the scale-free networks by synaptic re-organization by the spike timing dependent synaptic plasticity (STDP), which is hardly achieved with conventional H…
In this review, we describe the singular success of attractor neural network models in describing how the brain maintains persistent activity states for working memory, error-corrects, and integrates noisy cues. We consider the mechanisms by which simple and forgetful units can organize to collectively generate dynamics on the long time-scales required for such computations. We discuss the myriad potential uses of attractor dynamics for computation in the brain, and showcase notable examples of brain systems in which inherently low-dimensional continuous attractor dynamics have been concretely…
The human brain is organized in functional modules. Such an organization presents a basic conundrum: modules ought to be sufficiently independent to guarantee functional specialization and sufficiently connected to bind multiple processors for efficient information transfer. It is commonly accepted that small-world architecture of short lengths and large local clustering may solve this problem. However, there is intrinsic tension between shortcuts generating small-worlds and the persistence of modularity; a global property unrelated to local clustering. Here, we present a possible solution to …
Many studies have found evidence that the brain operates at a critical point, a processus known as self-organized criticality. A recent paper found remarkable scalings suggestive of criticality in systems as different as neural cultures, anesthetized or awake brains. We point out here that the diversity of these states would question any claimed role of criticality in information processing. Furthermore, we show that two non-critical systems pass all the tests for criticality, a control that was not provided in the original article. We conclude that such false positives demonstrate that the pr…
The human brain is a complex network of interconnected brain regions organized into functional modules with distinct roles in cognition and behavior. An important question concerns the persistence and stability of these modules over the human lifespan. Here we use graph-theoretic analysis to algorithmically uncover the brain's intrinsic modular organization across multiple spatial scales ranging from small communities comprised of only a few brain regions to large communities made up of many regions. We find that at coarse scales modules become progressively more segregated, while at finer sca…
Networks of living neurons exhibit an avalanche mode of activity, experimentally found in organotypic cultures. Moreover, experimental studies of morphology indicate that neurons develop a network of small-world-like connections, with the possibility of very high connectivity degree. Here we study a recent model based on self-organized criticality, which consists of an electrical network with threshold firing and activity-dependent synapse strengths. We study the model on a scale-free network, the Apollonian network, which presents many features of neuronal systems. The system exhibits a power…
The human brain is organized into large-scale functional modules that have been shown to evolve in childhood and adolescence. However, it remains unknown whether structural brain networks are similarly refined during development, potentially allowing for improvements in executive function. In a sample of 882 participants (ages 8-22) who underwent diffusion imaging as part of the Philadelphia Neurodevelopmental Cohort, we demonstrate that structural network modules become more segregated with age, with weaker connections between modules and stronger connections within modules. Evolving modular …
Networks and systems which exhibit brain-like behavior can analyze information from intrinsically noisy and unstructured data with very low power consumption. Such characteristics arise due to the critical nature and complex interconnectivity of the brain and its neuronal network. We demonstrate that a system comprising of multilayer hexagonal Boron Nitride (hBN) films contacted with Silver (Ag), that can uniquely host two different self-assembled networks, which are self-organized at criticality (SOC). This system shows bipolar resistive switching between high resistance (HRS) and low resista…
Neuropsychiatric disorders impact functional connectivity of the brain at the network level. The identification and statistical testing of disorder-related networks remains challenging. We propose novel methods to streamline the detection and testing of the hidden, disorder-related connectivity patterns as network-objects. We define an abnormal connectome subnetwork as a network-object that includes three classes: nodes of brain areas, edges representing brain connectomic features, and an organized graph topology formed by these nodes and edges. Comparing to the conventional statistical method…
Living neural networks in our brains autonomously self-organize into large, complex architectures during early development to result in an organized and functional organic computational device. A key mechanism that enables the formation of complex architecture in the developing brain is the emergence of traveling spatio-temporal waves of neuronal activity across the growing brain. Inspired by this strategy, we attempt to efficiently self-organize large neural networks with an arbitrary number of layers into a wide variety of architectures. To achieve this, we propose a modular tool-kit in the …
Neuronal firing activities have attracted a lot of attention since a large population of spatiotemporal patterns in the brain is the basis for adaptive behavior and can also reveal the signs for various neurological disorders including Alzheimer's, schizophrenia, epilepsy and others. Here, we study the dynamics of a simple neuronal network using different sets of settings on a neuromorphic chip. We observed three different types of collective neuronal firing activities, which agree with the clinical data taken from the brain. We constructed a brain phase diagram and showed that within the weak…
A formula for an average connectivity between cortical areas in mammals is derived. Based on comparative neuroanatomical data, it is found, surprisingly, that this connectivity is either only weakly dependent or independent of brain size. It is discussed how this formula can be used to estimate the average length of axons in white matter. Other allometric relations, such as cortical patches and area sizes vs. brain size, are also provided. Finally, some functional implications, with an emphasis on efficient cortical computation, are discussed as well.
The organization in brain networks shows highly modular features with weak inter-modular interaction. The topology of the networks involves emergence of modules and sub-modules at different levels of constitution governed by fractal laws. The modular organization, in terms of modular mass, inter-modular, and intra-modular interaction, also obeys fractal nature. The parameters which characterize topological properties of brain networks follow one parameter scaling theory in all levels of network structure which reveals the self-similar rules governing the network structure. The calculated fract…
The human brain has been studied at multiple scales, from neurons, circuits, areas with well defined anatomical and functional boundaries, to large-scale functional networks which mediate coherent cognition. In a recent work, we addressed the problem of the hierarchical organization in the brain through network analysis. Our analysis identified functional brain modules of fractal structure that were inter-connected in a small-world topology. Here, we provide more details on the use of network science tools to elaborate on this behavior. We indicate the importance of using percolation theory to…
Artificial intelligence (AI) is a fast-growing field focused on modeling and machine implementation of various cognitive functions with an increasing number of applications in computer vision, text processing, robotics, neurotechnology, bio-inspired computing and others. In this chapter, we describe how AI methods can be applied in the context of intracranial electroencephalography (iEEG) research. IEEG data is unique as it provides extremely high-quality signals recorded directly from brain tissue. Applying advanced AI models to these data carries the potential to further our understanding of…
Emergent Language Symbolic Autoencoder (ELSA) with Weak Supervision to Model Hierarchical Brain Networks
Johns Hopkins University（约翰斯·霍普金斯大学）；Microsoft Health AI（微软健康人工智能）；Johns Hopkins School of Medicine（约翰斯·霍普金斯大学医学院）；Malone Center for Engineering in Healthcare, Johns Hopkins University（约翰斯·霍普金斯大学马龙医疗工程中心）
Synfire chains are thought to underlie precisely-timed sequences of spikes observed in various brain regions and across species. How they are formed is not understood. Here we analyze self-organization of synfire chains through the spike-timing dependent plasticity (STDP) of the synapses, axon remodeling, and potentiation decay of synaptic weights in networks of neurons driven by noisy external inputs and subject to dominant feedback inhibition. Potentiation decay is the gradual, activity-independent reduction of synaptic weights over time. We show that potentiation decay enables a dynamic and…
Flexibility and Invariance of Object Representations in the Human Brain
Neuroscience Institute, Carnegie Mellon University（卡内基梅隆大学神经科学研究所）；Department of Psychology, Carnegie Mellon University（卡内基梅隆大学心理学系）；Department of Machine Learning, Carnegie Mellon University（卡内基梅隆大学机器学习系）；Department of Neurosurgery, University of Rochester Medical Center（罗切斯特大学医学中心神经外科）
This explanation of what a brain is and does rests on informational first principles, because information theory, like its parent theory thermodynamics, is mathematically sacrosanct, itself resting on real-valued probability.Just as thermodynamics has enabled hyper-potent physical technologies from the internal combustion engine to the hydrogen bomb, so information theory has enabled hyper-persuasive technologies, from color television to addictive video games. Only a theory of what a brain is and does based on those same principles makes legible and transparent the mechanisms by which such hy…
We use the Allen Gene Expression Atlas (AGEA) and the OMA ortholog dataset to investigate the evolution of mouse-brain neuroanatomy from the standpoint of the molecular evolution of brain-specific genes. For each such gene, using the phylogenetic tree for all fully sequenced species and the presence of orthologs of the gene in these species, we construct and assign a discrete measure of evolutionary age. The gene expression profile of all gene of similar age, relative to the average gene expression profile, distinguish regions of the brain that are over-represented in the corresponding evoluti…
Contemporary neuroscience has embraced network science to study the complex and self-organized structure of the human brain; one of the main outstanding issues is that of inferring from measure data, chiefly functional Magnetic Resonance Imaging (fMRI), the so-called effective connectivity in brain networks, that is the existing interactions among neuronal populations. This inverse problem is complicated by the fact that the BOLD (Blood Oxygenation Level Dependent) signal measured by fMRI represent a dynamic and nonlinear transformation (the hemodynamic response) of neuronal activity. In this …
A pressing scientific challenge is to understand how brains work. Of particular interest is the neocortex,the part of the brain that is especially large in humans, capable of handling a wide variety of tasks including visual, auditory, language, motor, and abstract processing. These functionalities are processed in different self-organized regions of the neocortical sheet, and yet the anatomical structure carrying out the processing is relatively uniform across the sheet. We are at a loss to explain, simulate, or understand such a multi-functional homogeneous sheet-like computational structure…
Self-organized criticality has been proposed to be a universal mechanism for the emergence of scale-free dynamics in many complex systems, and possibly in the brain. While such scale-free patterns were identified experimentally in many different types of neural recordings, the biological principles behind their emergence remained unknown. Utilizing different network models and motivated by experimental observations, synaptic plasticity was proposed as a possible mechanism to self-organize brain dynamics towards a critical point. In this review, we discuss how various biologically plausible pla…
University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）；Korea University（高丽大学）
The human brains are organized into hierarchically modular networks facilitating efficient and stable information processing and supporting diverse cognitive processes during the course of development. While the remarkable reconfiguration of functional brain network has been firmly established in early life, all these studies investigated the network development from a "single-scale" perspective, which ignore the richness engendered by its hierarchical nature. To fill this gap, this paper leveraged a longitudinal infant resting-state functional magnetic resonance imaging dataset from birth to …
ETIS Lab, UMR8051, CY Cergy Paris University, ENSEA, CNRS（ETIS实验室，UMR8051，塞尔吉巴黎大学，ENSEA，法国国家科学研究中心）
In order to keep trace of information and grow up, the infant brain has to resolve the problem about where old information is located and how to index new ones. We propose that the immature prefrontal cortex (PFC) use its primary functionality of detecting hierarchical patterns in temporal signals as a second purpose to organize the spatial ordering of the cortical networks in the developing brain itself. Our hypothesis is that the PFC detects the hierarchical structure in temporal sequences in the form of ordinal patterns and use them to index information hierarchically in different parts of …
University of Liverpool（利物浦大学）；Institute of Ageing and Chronic Diseases（衰老与慢性病研究所）
Here we provide evidence that the fundamental basis of nervous communication is derived from a pressure pulse/soliton capable of computation with sufficient temporal precision to overcome any processing errors. Signalling and computing within the nervous system are complex and different phenomena. Action potentials are plastic and this makes the action potential peak an inappropriate fixed point for neural computation, but the action potential threshold is suitable for this purpose. Furthermore, neural models timed by spiking neurons operate below the rate necessary to overcome processing erro…
What does it take for a system, biological or not, to have goals? Here, this question is approached in the context of in silico artificial evolution. By examining the informational and causal properties of artificial organisms ('animats') controlled by small, adaptive neural networks (Markov Brains), this essay discusses necessary requirements for intrinsic information, autonomy, and meaning. The focus lies on comparing two types of Markov Brains that evolved in the same simple environment: one with purely feedforward connections between its elements, the other with an integrated set of elemen…
Frankfurt Institute for Advanced Studies（法兰克福高等研究院）；Zurich University of Applied Sciences（苏黎世应用科学大学）；University of Zurich（苏黎世大学）；ETH Zurich（苏黎世联邦理工学院）；European Centre for Living Technology（欧洲生命技术中心）
Introduction: In contrast to current AI technology, natural intelligence -- the kind of autonomous intelligence that is realized in the brains of animals and humans to attain in their natural environment goals defined by a repertoire of innate behavioral schemata -- is far superior in terms of learning speed, generalization capabilities, autonomy and creativity. How are these strengths, by what means are ideas and imagination produced in natural neural networks? Methods: Reviewing the literature, we put forward the argument that both our natural environment and the brain are of low complexity,…
Recent developments in network neuroscience have highlighted the importance of developing techniques for analyzing and modeling brain networks. A particularly powerful approach for studying complex neural systems is to formulate generative models that use wiring rules to synthesize networks closely resembling the topology of a given connectome. Successful models can highlight the principles by which a network is organized (identify structural features that arise from wiring rules versus those that emerge) and potentially uncover the mechanisms by which it grows and develops. Previous research …
University of Rochester（罗切斯特大学）；Duke University（杜克大学）；New York University（纽约大学）；Gatsby Computational Neuroscience Unit（盖茨比计算神经科学单元）；University College London（伦敦大学学院）；Neuroscience Institute（神经科学研究所）；Carnegie Mellon University（卡内基梅隆大学）；Center for Neuroscience and Artificial Intelligence（神经科学与人工智能中心）；Baylor College of Medicine（贝勒医学院）；Rice University（莱斯大学）
Open problems abound in the theory of complex networks, which has found successful application to diverse fields of science. With the aim of further advancing the understanding of the brain's functional connectivity, we propose to evaluate a network metric which we term the geodesic entropy. This entropy, in a way that can be made precise, quantifies the Shannon entropy of the distance distribution to a specific node from all other nodes. Measurements of geodesic entropy allow for the characterization of the structural information of a network that takes into account the distinct role of each …
Experimental fMRI studies have shown that spontaneous brain activity i.e. in the absence of any external input, exhibit complex spatial and temporal patterns of co-activity between segregated brain regions. These so-called large-scale resting-state functional connectivity networks represent dynamically organized neural assemblies interacting with each other in a complex way. It has been suggested that looking at the dynamical properties of complex patterns of brain functional co-activity may reveal neural mechanisms underlying the dynamic changes in functional interactions. Here, we examine ho…
Anatomic connections between brain areas affect information flow between neuronal circuits and the synchronization of neuronal activity. However, such structural connectivity does not coincide with effective connectivity, related to the more elusive question "Which areas cause the present activity of which others?". Effective connectivity is directed and depends flexibly on contexts and tasks. Here we show that a dynamic effective connectivity can emerge from transitions in the collective organization of coherent neural activity. Integrating simulation and semi-analytic approaches, we study me…
The understanding of neural activity patterns is fundamentally linked to an understanding of how the brain's network architecture shapes dynamical processes. Established approaches rely mostly on deviations of a given network from certain classes of random graphs. Hypotheses about the supposed role of prominent topological features (for instance, the roles of modularity, network motifs, or hierarchical network organization) are derived from these deviations. An alternative strategy could be to study deviations of network architectures from regular graphs (rings, lattices) and consider the impl…
An approach to information processing based on the excitation of patterns of activity by non-linear active resonators in response to their input patterns is proposed. Arguments are presented to show that any computation performed by a conventional Turing machine-based computer, called T-machine in this paper, could also be performed by the pattern excitation-based machine, which will be called P-machine. A realization of this processing scheme by neural networks is discussed. In this realization, the role of the resonators is played by neural pattern excitation networks, which are the neural c…
