Fifty years ago, John von Neumann compared the architecture of the brain with that of computers that he invented and which is still in use today. In those days, the organisation of computers was based on concepts of brain organisation. Here, we give an update on current results on the global organisation of neural systems. For neural systems, we outline how the spatial and topological architecture of neuronal and cortical networks facilitates robustness against failures, fast processing, and balanced network activation. Finally, we discuss mechanisms of self-organization for such architectures. After all, the organization of the brain might again inspire computer architecture.
Neuroscience Institute（神经科学研究所）；Department of Machine Learning（机器学习系）；Carnegie Mellon University（卡内基梅隆大学）
Neural networks models for NLP are typically implemented without the explicit encoding of language rules and yet they are able to break one performance record after another. This has generated a lot of research interest in interpreting the representations learned by these networks. We propose here a novel interpretation approach that relies on the only processing system we have that does understand language: the human brain. We use brain imaging recordings of subjects reading complex natural text to interpret word and sequence embeddings from 4 recent NLP models - ELMo, USE, BERT and Transform…
Universidad Autónoma de Madrid（马德里自治大学）；Sano - Centre for Computational Personalised Medicine（Sano计算个性化医学中心）
Synaptic plasticity allows cortical circuits to learn new tasks and to adapt to changing environments. How do cortical circuits use plasticity to acquire functions such as decision-making or working memory? Neurons are connected in complex ways, forming recurrent neural networks, and learning modifies the strength of their connections. Moreover, neurons communicate emitting brief discrete electric signals. Here we describe how to train recurrent neural networks in tasks like those used to train animals in neuroscience laboratories, and how computations emerge in the trained networks. Surprisin…
In recent years, several studies have provided insight on the functioning of the brain which consists of neurons and form networks via interconnection among them by synapses. Neural networks are formed by interconnected systems of neurons, and are of two types, namely, the Artificial Neural Network (ANNs) and Biological Neural Network (interconnected nerve cells). The ANNs are computationally influenced by human neurons and are used in modelling neural systems. The reasoning foundations of ANNs have been useful in anomaly detection, in areas of medicine such as instant physician, electronic no…
Frankfurt Institute for Advanced Studies（法兰克福高等研究院）；Zurich University of Applied Sciences（苏黎世应用科学大学）；University of Zurich（苏黎世大学）；ETH Zurich（苏黎世联邦理工学院）；European Centre for Living Technology（欧洲生命技术中心）
Introduction: In contrast to current AI technology, natural intelligence -- the kind of autonomous intelligence that is realized in the brains of animals and humans to attain in their natural environment goals defined by a repertoire of innate behavioral schemata -- is far superior in terms of learning speed, generalization capabilities, autonomy and creativity. How are these strengths, by what means are ideas and imagination produced in natural neural networks? Methods: Reviewing the literature, we put forward the argument that both our natural environment and the brain are of low complexity,…
We review quantitative methods and software developed to analyze genome-scale, brain-wide spatially-mapped gene-expression data. We expose new methods based on the underlying high-dimensional geometry of voxel space and gene space, and on simulations of the distribution of co-expression networks of a given size. We apply them to the Allen Atlas of the adult mouse brain, and to the co-expression network of a set of genes related to nicotine addiction retrieved from the NicSNP database. The computational methods are implemented in {\ttfamily{BrainGeneExpressionAnalysis}}, a Matlab toolbox availa…
Finding relevant information from large document collections such as the World Wide Web is a common task in our daily lives. Estimation of a user's interest or search intention is necessary to recommend and retrieve relevant information from these collections. We introduce a brain-information interface used for recommending information by relevance inferred directly from brain signals. In experiments, participants were asked to read Wikipedia documents about a selection of topics while their EEG was recorded. Based on the prediction of word relevance, the individual's search intent was modeled…
University of Tokyo（东京大学）；University of Oxford（牛津大学）；ATR（国际电气通信基础技术研究所）；AIST（产业技术综合研究所）
Multi-regional interaction among neuronal populations underlies the brain's processing of rich sensory information in our daily lives. Recent neuroscience and neuroimaging studies have increasingly used naturalistic stimuli and experimental design to identify such realistic sensory computation in the brain. However, existing methods for cross-areal interaction analysis with dimensionality reduction, such as reduced-rank regression and canonical correlation analysis, have limited applicability and interpretability in naturalistic settings because they usually do not appropriately 'demix' neural…
The deep neural nets of modern artificial intelligence (AI) have not achieved defining features of biological intelligence, including abstraction, causal learning, and energy-efficiency. While scaling to larger models has delivered performance improvements for current applications, more brain-like capacities may demand new theories, models, and methods for designing artificial learning systems. Here, we argue that this opportunity to reassess insights from the brain should stimulate cooperation between AI research and theory-driven computational neuroscience (CN). To motivate a brain basis of …
Progress in natural language processing (NLP) models that estimate representations of word sequences has recently been leveraged to improve the understanding of language processing in the brain. However, these models have not been specifically designed to capture the way the brain represents language meaning. We hypothesize that fine-tuning these models to predict recordings of brain activity of people reading text will lead to representations that encode more brain-activity-relevant language information. We demonstrate that a version of BERT, a recently introduced and powerful language model,…
The relationship between brains and computers is often taken to be merely metaphorical. However, genuine computational systems can be implemented in virtually any media; thus, one can take seriously the view that brains literally compute. But without empirical criteria for what makes a physical system genuinely a computational one, computation remains a matter of perspective, especially for natural systems (e.g., brains) that were not explicitly designed and engineered to be computers. Considerations from real examples of physical computers-both analog and digital, contemporary and historical-…
Princeton University（普林斯顿大学）；Max Planck Institute for Software Systems（马克斯·普朗克软件系统研究所）；Carnegie Mellon University（卡内基梅隆大学）；Google Research（谷歌研究院）
To study information processing in the brain, neuroscientists manipulate experimental stimuli while recording participant brain activity. They can then use encoding models to find out which brain "zone" (e.g. which region of interest, volume pixel or electrophysiology sensor) is predicted from the stimulus properties. Given the assumptions underlying this setup, when stimulus properties are predictive of the activity in a zone, these properties are understood to cause activity in that zone. In recent years, researchers have used neural networks to construct representations that capture the div…
Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University（西安交通大学人工智能与机器人研究所）
Neural encoding and decoding, which aim to characterize the relationship between stimuli and brain activities, have emerged as an important area in cognitive neuroscience. Traditional encoding models, which focus on feature extraction and mapping, consider the brain as an input-output mapper without inner states. In this work, inspired by the fact that human brain acts like a state machine, we proposed a novel encoding framework that combines information from both the external world and the inner state to predict brain activity. The framework comprises two parts: forward encoding model that de…
Brain-computer interfaces (BCIs) are enabling a range of new possibilities and routes for augmenting human capability. Here, we propose BCIs as a route towards forms of computation, i.e. computational imaging, that blend the brain with external silicon processing. We demonstrate ghost imaging of a hidden scene using the human visual system that is combined with an adaptive computational imaging scheme. This is achieved through a projection pattern `carving' technique that relies on real-time feedback from the brain to modify patterns at the light projector, thus enabling more efficient and hig…
Reconstructing perceived images from human brain activity monitored by functional magnetic resonance imaging (fMRI) is hard, especially for natural images. Existing methods often result in blurry and unintelligible reconstructions with low fidelity. In this study, we present a novel approach for enhanced image reconstruction, in which existing methods for object decoding and image reconstruction are merged together. This is achieved by conditioning the reconstructed image to its decoded image category using a class-conditional generative adversarial network and neural style transfer. The resul…
Facebook AI Research（Facebook人工智能研究院）；Université Paris-Saclay（巴黎萨克雷大学）；Inria（法国国家信息与自动化研究所）；CEA（法国原子能和替代能源委员会）；École normale supérieure, PSL University（巴黎高等师范学院，PSL大学）；CNRS（法国国家科学研究中心）
Deep learning has recently made remarkable progress in natural language processing. Yet, the resulting algorithms remain far from competing with the language abilities of the human brain. Predictive coding theory offers a potential explanation to this discrepancy: while deep language algorithms are optimized to predict adjacent words, the human brain would be tuned to make long-range and hierarchical predictions. To test this hypothesis, we analyze the fMRI brain signals of 304 subjects each listening to 70min of short stories. After confirming that the activations of deep language algorithms …
University of Liverpool（利物浦大学）；Institute of Ageing and Chronic Diseases（衰老与慢性病研究所）
Here we provide evidence that the fundamental basis of nervous communication is derived from a pressure pulse/soliton capable of computation with sufficient temporal precision to overcome any processing errors. Signalling and computing within the nervous system are complex and different phenomena. Action potentials are plastic and this makes the action potential peak an inappropriate fixed point for neural computation, but the action potential threshold is suitable for this purpose. Furthermore, neural models timed by spiking neurons operate below the rate necessary to overcome processing erro…
Fully automated decoding of human activities and intentions from direct neural recordings is a tantalizing challenge in brain-computer interfacing. Most ongoing efforts have focused on training decoders on specific, stereotyped tasks in laboratory settings. Implementing brain-computer interfaces (BCIs) in natural settings requires adaptive strategies and scalable algorithms that require minimal supervision. Here we propose an unsupervised approach to decoding neural states from human brain recordings acquired in a naturalistic context. We demonstrate our approach on continuous long-term electr…
A formula for an average connectivity between cortical areas in mammals is derived. Based on comparative neuroanatomical data, it is found, surprisingly, that this connectivity is either only weakly dependent or independent of brain size. It is discussed how this formula can be used to estimate the average length of axons in white matter. Other allometric relations, such as cortical patches and area sizes vs. brain size, are also provided. Finally, some functional implications, with an emphasis on efficient cortical computation, are discussed as well.
Understanding neurocognitive computations will require not just localizing cognitive information distributed throughout the brain but also determining how that information got there. We review recent advances in linking empirical and simulated brain network organization with cognitive information processing. Building on these advances, we offer a new framework for understanding the role of connectivity in cognition - network coding (encoding/decoding) models. These models utilize connectivity to specify the transfer of information via neural activity flow processes, successfully predicting the…
Michigan State University（密歇根州立大学）；California Institute of Technology（加州理工学院）；University of Pennsylvania（宾夕法尼亚大学）；University of Wisconsin（威斯康星大学）；Indiana University（印第安纳大学）
Markov Brains are a class of evolvable artificial neural networks (ANN). They differ from conventional ANNs in many aspects, but the key difference is that instead of a layered architecture, with each node performing the same function, Markov Brains are networks built from individual computational components. These computational components interact with each other, receive inputs from sensors, and control motor outputs. The function of the computational components, their connections to each other, as well as connections to sensors and motors are all subject to evolutionary optimization. Here w…
For complex life to evolve, a sophisticated nervous system for handling its complexities was fundamental. The demand resulted in the emergence of brain's computational facility, the neuronal network. This facet of the brain is attested solidly by its inspired scientific computational neural nets which (mathematically) resolve and solve many complex problems. The presumptive general semblance of the computational operation between the two systems allows for the inference that the process in brain's neural domain also renders complexities for solution, as sets of parametric equations, like the b…
University of Oxford（牛津大学）；Max Planck Institute for Psycholinguistics（马克斯·普朗克心理语言学研究所）；Radboud University（拉德堡德大学）
Recent artificial neural networks that process natural language achieve unprecedented performance in tasks requiring sentence-level understanding. As such, they could be interesting models of the integration of linguistic information in the human brain. We review works that compare these artificial language models with human brain activity and we assess the extent to which this approach has improved our understanding of the neural processes involved in natural language comprehension. Two main results emerge. First, the neural representation of word meaning aligns with the context-dependent, de…
As Brain-computer interface (BCI) technology develops it is likely it may be incorporated into protocols that complement and supplement existing movements of the user. Two possible scenarios for such a control could be: the increasing interest to control artificial supernumerary prosthetics, or in cases following brain injury where BCI can be incorporated alongside residual movements to recover ability. In this study we explore the extent to which the human motor cortex is able to concurrently control movements via a BCI and overtly executed movements. Crucially both movement types are driven …
PLA strategy support force information engineering university（中国人民解放军战略支援部队信息工程大学）
In the visual decoding domain, visually reconstructing presented images given the corresponding human brain activity monitored by functional magnetic resonance imaging (fMRI) is difficult, especially when reconstructing viewed natural images. Visual reconstruction is a conditional image generation on fMRI data and thus generative adversarial network (GAN) for natural image generation is recently introduced for this task. Although GAN-based methods have greatly improved, the fidelity and naturalness of reconstruction are still unsatisfactory due to the small number of fMRI data samples and the …
This article serves as a supplement to the recently published call for participation in a Research Topic [1] that is timed to commemorate the 75th anniversary of Shannon's pioneering 1948 paper [2]. Here, we include some citations of key and relevant literature, which reflect our opinions/perspectives on the proposed topic, and serve as guidance to potential submissions.
Since the beginning of information processing by electronic components, the nervous system has served as a metaphor for the organization of computational primitives. Brain-inspired computing today encompasses a class of approaches ranging from using novel nano-devices for computation to research into large-scale neuromorphic architectures, such as TrueNorth, SpiNNaker, BrainScaleS, Tianjic, and Loihi. While implementation details differ, spiking neural networks - sometimes referred to as the third generation of neural networks - are the common abstraction used to model computation with such sy…
The Whole Brain Architecture Initiative（全脑架构倡议组织）；The University of Tokyo（东京大学）；RIKEN（理化学研究所）
The vastness of the design space created by the combination of a large number of computational mechanisms, including machine learning, is an obstacle to creating an artificial general intelligence (AGI). Brain-inspired AGI development, in other words, cutting down the design space to look more like a biological brain, which is an existing model of a general intelligence, is a promising plan for solving this problem. However, it is difficult for an individual to design a software program that corresponds to the entire brain because the neuroscientific data required to understand the architectur…
Many recent efforts in computational modeling of macro-scale brain dynamics have begun to take a data-driven approach by incorporating structural and/or functional information derived from subject data. Here, we discuss recent work using personalized brain network models to study structure-function relationships in human brains. We describe the steps necessary to build such models and show how this computational approach can provide previously unobtainable information through the ability to perform virtual experiments. Finally, we present examples of how personalized brain network models can b…
Accurate Decoding of Natural Sentences from Non-Invasive Brain Recordings
Meta AI；École Normale Supérieure（高等师范学院）；Université PSL（巴黎文理大学）；CNRS（法国国家科学研究中心）；Hospital Foundation Adolphe de Rothschild（阿道夫·德·罗斯柴尔德医院基金会）；Univ. Lille（里尔大学）；Basque Center on Cognition, Brain and Language（巴斯克认知、大脑与语言中心）；Paris Cité University（巴黎城市大学）；Inria（法国国家信息与自动化研究所）；Université Paris-Saclay（巴黎萨克雷大学）；INSERM（法国国家健康与医学研究院）；CEA（法国原子能和替代能源委员会）
Tokyo Institute of Technology（东京工业大学）；National Institute of Advanced Industrial Science and Technology（产业技术综合研究所）；AIST-Tokyo Tech Real World Big-Data Computation Open Innovation Laboratory（AIST-东京工业大学真实世界大数据计算开放创新实验室）
With the advent of brain imaging techniques and machine learning tools, much effort has been devoted to building computational models to capture the encoding of visual information in the human brain. One of the most challenging brain decoding tasks is the accurate reconstruction of the perceived natural images from brain activities measured by functional magnetic resonance imaging (fMRI). In this work, we survey the most recent deep learning methods for natural image reconstruction from fMRI. We examine these methods in terms of architectural design, benchmark datasets, and evaluation metrics …
The relationship between brain structure and function has been probed using a variety of approaches, but how the underlying structural connectivity of the human brain drives behavior is far from understood. To investigate the effect of anatomical brain organization on human task performance, we use a data-driven computational modeling approach and explore the functional effects of naturally occurring structural differences in brain networks. We construct personalized brain network models by combining anatomical connectivity estimated from diffusion spectrum imaging of individual subjects with …
How do humans and other animals learn new tasks? A wave of brain recording studies has investigated how neural representations change during task learning, with a focus on how tasks can be acquired and coded in ways that minimise mutual interference. We review recent work that has explored the geometry and dimensionality of neural task representations in neocortex, and computational models that have exploited these findings to understand how the brain may partition knowledge between tasks. We discuss how ideas from machine learning, including those that combine supervised and unsupervised lear…
Linguistics and Human Brain: A Perspective of Computational Neuroscience
Institute of AI and Robotics, College of Intelligent Robotics and Advanced Manufacturing, Fudan University（人工智能与机器人研究院，智能机器人与先进制造学院，复旦大学）；Department of Language Science and Technology, The Hong Kong Polytechnic University（语言科学与技术系，香港理工大学）；Department of Computing, The Hong Kong Polytechnic University（计算系，香港理工大学）
In this review, we describe the singular success of attractor neural network models in describing how the brain maintains persistent activity states for working memory, error-corrects, and integrates noisy cues. We consider the mechanisms by which simple and forgetful units can organize to collectively generate dynamics on the long time-scales required for such computations. We discuss the myriad potential uses of attractor dynamics for computation in the brain, and showcase notable examples of brain systems in which inherently low-dimensional continuous attractor dynamics have been concretely…
University of Wisconsin-Madison（威斯康星大学麦迪逊分校）；Vanderbilt University（范德堡大学）；University of Chicago（芝加哥大学）
In many human brain network studies, we do not have sufficient number (n) of images relative to the number (p) of voxels due to the prohibitively expensive cost of scanning enough subjects. Thus, brain network models usually suffer the small-n large-p problem. Such a problem is often remedied by sparse network models, which are usually solved numerically by optimizing L1-penalties. Unfortunately, due to the computational bottleneck associated with optimizing L1-penalties, it is not practical to apply such methods to construct large-scale brain networks at the voxel-level. In this paper, we pro…
In the past few years, significant advancements were made in reconstruction of observed natural images from fMRI brain recordings using deep-learning tools. Here, for the first time, we show that dense 3D depth maps of observed 2D natural images can also be recovered directly from fMRI brain recordings. We use an off-the-shelf method to estimate the unknown depth maps of natural images. This is applied to both: (i) the small number of images presented to subjects in an fMRI scanner (images for which we have fMRI recordings - referred to as "paired" data), and (ii) a very large number of natura…
National Digital Switching System Engineering and Technological Research Center（国家数字交换系统工程技术研究中心）；University of Electronic Science and Technology of China（电子科技大学）
In recent years, research on decoding brain activity based on functional magnetic resonance imaging (fMRI) has made remarkable achievements. However, constraint-free natural image reconstruction from brain activity is still a challenge. The existing methods simplified the problem by using semantic prior information or just reconstructing simple images such as letters and digitals. Without semantic prior information, we present a novel method to reconstruct nature images from fMRI signals of human visual cortex based on the computation model of convolutional neural network (CNN). Firstly, we ex…
Large efforts are currently under way to systematically map functional connectivity between all pairs of millimeter-scale brain regions using big volumes of neuroimaging data. Functional magnetic resonance imaging (fMRI) can produce these functional connectomes, however, large amounts of data and lengthy computation times add important overhead to this task. Previous work has demonstrated that fMRI data admits a sparse representation in the form a discrete point-process containing sufficient information for the efficient estimation of functional connectivity between all pairs of voxels. In thi…
Quantifying the cost of network computations to unpack structure-function relationships in the brain
University of Pennsylvania（宾夕法尼亚大学）；Cornell University（康奈尔大学）；Massachusetts General Hospital, Harvard Medical School（哈佛医学院马萨诸塞州总医院）；University of California, Irvine（加州大学尔湾分校）；Yale University（耶鲁大学）
Several popular Transformer based language models have been found to be successful for text-driven brain encoding. However, existing literature leverages only pretrained text Transformer models and has not explored the efficacy of task-specific learned Transformer representations. In this work, we explore transfer learning from representations learned for ten popular natural language processing tasks (two syntactic and eight semantic) for predicting brain responses from two diverse datasets: Pereira (subjects reading sentences from paragraphs) and Narratives (subjects listening to the spoken s…
This paper proposes a novel topological learning framework that integrates networks of different sizes and topology through persistent homology. Such challenging task is made possible through the introduction of a computationally efficient topological loss. The use of the proposed loss bypasses the intrinsic computational bottleneck associated with matching networks. We validate the method in extensive statistical simulations to assess its effectiveness when discriminating networks with different topology. The method is further demonstrated in a twin brain imaging study where we determine if b…
Objective: The aim of this study is to propose an adaptive scheme embedded into an open-source environment for the estimation of the neural activation extent during deep brain stimulation and to investigate the feasibility of approximating the neural activation extent by thresholds of the field solution. Methods: Open-source solutions for solving the field equation in volume conductor models of deep brain stimulation and computing the neural activation are embedded into a Python package to estimate the neural activation dependent on the dielectric tissue properties and axon parameters by emplo…
The comparison of observed brain activity with the statistics generated by artificial intelligence systems is useful to probe brain functional organization under ecological conditions. Here we study fMRI activity in ten subjects watching color natural movies and compute deep representations of these movies with an architecture that relies on optical flow and image content. The association of activity in visual areas with the different layers of the deep architecture displays complexity-related contrasts across visual areas and reveals a striking foveal/peripheral dichotomy.
