How to optimize neuroscience data utilization and experiment design for advancing brain models of visual and linguistic cognition?
Greta Tuckute, Dawn Finzi, Eshed Margalit, Joel Zylberberg, SueYeon Chung, Alona Fyshe, Evelina Fedorenko, Nikolaus Kriegeskorte, Jacob Yates, Kalanit Grill-Spector, Kohitij Kar
In recent years, neuroscience has made significant progress in building large-scale artificial neural network (ANN) models of brain activity and behavior. However, there is no consensus on the most efficient ways to collect data and design experiments to develop the next generation of models. This article explores the controversial opinions that have emerged on this topic in the domain of vision and language. Specifically, we address two critical points. First, we weigh the pros and cons of using qualitative insights from empirical results versus raw experimental data to train models. Second, we consider model-free (intuition-based) versus model-based approaches for data collection, specifically experimental design and stimulus selection, for optimal model development. Finally, we consider the challenges of developing a synergistic approach to experimental design and model building, including encouraging data and model sharing and the implications of iterative additions to existing models. The goal of the paper is to discuss decision points and propose directions for both experimenters and model developers in the quest to understand the brain.
To learn how cognition is implemented in the brain, we must build computational models that can perform cognitive tasks, and test such models with brain and behavioral experiments. Cognitive science has developed computational models of human cognition, decomposing task performance into computational components. However, its algorithms still fall short of human intelligence and are not grounded in neurobiology. Computational neuroscience has investigated how interacting neurons can implement component functions of brain computation. However, it has yet to explain how those components interact …
A major shift is happening within neurophysiology: a population doctrine is drawing level with the single-neuron doctrine that has long dominated the field. Population-level ideas have so far had their greatest impact in motor neuroscience, but they hold great promise for resolving open questions in cognition as well. Here, we codify the population doctrine and survey recent work that leverages this view to specifically probe cognition. Our discussion is organized around five core concepts that provide a foundation for population-level thinking: (1) state spaces, (2) manifolds, (3) coding dime…
Physical symbol systems are needed for open-ended cognition. A good way to understand physical symbol systems is by comparison of thought to chemistry. Both have systematicity, productivity and compositionality. The state of the art in cognitive architectures for open-ended cognition is critically assessed. I conclude that a cognitive architecture that evolves symbol structures in the brain is a promising candidate to explain open-ended cognition. Part 2 of the paper presents such a cognitive architecture.
A pervasive research protocol of cognitive neuroscience is to train subjects to perform deliberately designed experiments and record brain activity simultaneously, aiming to understand the brain mechanism underlying cognition. However, how the results of this protocol can be applied in technology is seldom discussed. Here, I review the studies on time processing of the brain as examples of this protocol, as well as two main application areas of neuroscience (neuroengineering and brain-inspired artificial intelligence). Time processing is an indispensable dimension of cognition; time is also an…
Connectomics and network neuroscience offer quantitative scientific frameworks for modeling and analyzing networks of structurally and functionally interacting neurons, neuronal populations, and macroscopic brain areas. This shift in perspective and emphasis on distributed brain function has provided fundamental insight into the role played by the brain's network architecture in cognition, disease, development, and aging. In this chapter, we review the core concepts of human connectomics at the macroscale. From the construction of networks using functional and diffusion MRI data, to their subs…
In this paper, I examine what I refer to as the spike doctrine, which is the generally held belief in neuroscience that information in the brain is encoded by sequences of neural action potentials. I present the argument that specific neurochemicals, and not spikes, are the elementary units of information in the brain. I outline several predictions that arise from this interpretation, relate them to results in the current research literature, and show how they address some open questions.
Cognitive neuroscience explores the mechanisms of cognition by studying its structural and functional brain correlates. Here, we report the first systematic review that assesses how information from structural and functional neuroimaging methods can be integrated to investigate the brain substrates of cognition. Web of Science and Scopus databases were searched for studies of healthy young adult populations that collected cognitive data, and structural and functional neuroimaging data. Five percent of screened studies met all inclusion criteria. Next, 54% of included studies related cognitive …
Cognitive function requires the coordination of neural activity across many scales, from neurons and circuits to large-scale networks. As such, it is unlikely that an explanatory framework focused upon any single scale will yield a comprehensive theory of brain activity and cognitive function. Modelling and analysis methods for neuroscience should aim to accommodate multiscale phenomena. Emerging research now suggests that multi-scale processes in the brain arise from so-called critical phenomena that occur very broadly in the natural world. Criticality arises in complex systems perched betwee…
Neuroscience is undergoing faster changes than ever before. Over 100 years our field qualitatively described and invasively manipulated single or few organisms to gain anatomical, physiological, and pharmacological insights. In the last 10 years neuroscience spawned quantitative big-sample datasets on microanatomy, synaptic connections, optogenetic brain-behavior assays, and high-level cognition. While growing data availability and information granularity have been amply discussed, we direct attention to a routinely neglected question: How will the unprecedented data richness shape data analys…
Building machines that learn and think like humans is essential not only for cognitive science, but also for computational neuroscience, whose ultimate goal is to understand how cognition is implemented in biological brains. A new cognitive computational neuroscience should build cognitive-level and neural- level models, understand their relationships, and test both types of models with both brain and behavioral data.
Neuroscience research is undergoing a minor revolution. Recent advances in machine learning and artificial intelligence (AI) research have opened up new ways of thinking about neural computation. Many researchers are excited by the possibility that deep neural networks may offer theories of perception, cognition and action for biological brains. This perspective has the potential to radically reshape our approach to understanding neural systems, because the computations performed by deep networks are learned from experience, not endowed by the researcher. If so, how can neuroscientists use dee…
This paper describes the outlines of a research program for understanding the cognitive-emotional brain, with an emphasis on the issue of dynamics: How can we study, characterize, and understand the neural underpinnings of cognitive-emotional behaviors as inherently dynamic processes? The framework embraces many of the central themes developed by Steve Grossberg in his extensive body of work in the past 50 years. By embracing head on the leitmotifs of dynamics, decentralized computation, emergence, selection and competition, and autonomy, it is proposed that a science of the mind-brain can be …
BITS Pilani K K Birla Goa Campus（Birla技术与科学学院皮拉尼分校KK Birla果阿校区）；New York University（纽约大学）；Amazon（亚马逊）
Reinforcement learning methods have recently been very successful at performing complex sequential tasks like playing Atari games, Go and Poker. These algorithms have outperformed humans in several tasks by learning from scratch, using only scalar rewards obtained through interaction with their environment. While there certainly has been considerable independent innovation to produce such results, many core ideas in reinforcement learning are inspired by phenomena in animal learning, psychology and neuroscience. In this paper, we comprehensively review a large number of findings in both neuros…
Justus-Liebig University Giessen（吉森尤斯利比希大学）；Columbia University（哥伦比亚大学）
Neural network models can now recognise images, understand text, translate languages, and play many human games at human or superhuman levels. These systems are highly abstracted, but are inspired by biological brains and use only biologically plausible computations. In the coming years, neural networks are likely to become less reliant on learning from massive labelled datasets, and more robust and generalisable in their task performance. From their successes and failures, we can learn about the computational requirements of the different tasks at which brains excel. Deep learning also provid…
The human brain is a complex network that supports mental function. The nascent field of network neuroscience applies tools from mathematics to neuroimaging data in the hopes of shedding light on cognitive function. A critical question arising from these empirical studies is how to modulate a human brain network to treat cognitive deficits or enhance mental abilities. While historically a number of tools have been employed to modulate mental states (such as cognitive behavioral therapy and brain stimulation), theoretical frameworks to guide these interventions - and to optimize them for clinic…
Mounting evidence in neuroscience suggests the possibility of neuronal representations that individual neurons serve as the substrates of different mental representations in a point-to-point way. Combined with associationism, it can potentially address a range of theoretical problems and provide a straightforward explanation for our cognition. However, this idea is merely a hypothesis with many questions unsolved. In this paper, I will bring up a new framework to defend the idea of neuronal representations. The strategy is from micro- to macro-level. Specifically, in the micro-level, I first p…
Cognitive neuroscience is enjoying rapid increase in extensive public brain-imaging datasets. It opens the door to large-scale statistical models. Finding a unified perspective for all available data calls for scalable and automated solutions to an old challenge: how to aggregate heterogeneous information on brain function into a universal cognitive system that relates mental operations/cognitive processes/psychological tasks to brain networks? We cast this challenge in a machine-learning approach to predict conditions from statistical brain maps across different studies. For this, we leverage…
Artificial intelligence (AI) is a fast-growing field focused on modeling and machine implementation of various cognitive functions with an increasing number of applications in computer vision, text processing, robotics, neurotechnology, bio-inspired computing and others. In this chapter, we describe how AI methods can be applied in the context of intracranial electroencephalography (iEEG) research. IEEG data is unique as it provides extremely high-quality signals recorded directly from brain tissue. Applying advanced AI models to these data carries the potential to further our understanding of…
The principled design and discovery of biologically- and physically-informed models of neuronal dynamics has been advancing since the mid-twentieth century. Recent developments in artificial intelligence (AI) have accelerated this progress. This review article gives a high-level overview of the approaches across different scales of organization and levels of abstraction. The studies covered in this paper include fundamental models in computational neuroscience, nonlinear dynamics, data-driven methods, as well as emergent practices. While not all of these models span the intersection of neurosc…
Although social neuroscience is concerned with understanding how the brain interacts with its social environment, prevailing research in the field has primarily considered the human brain in isolation, deprived of its rich social context. Emerging work in social neuroscience that leverages tools from network analysis has begun to pursue this issue, advancing knowledge of how the human brain influences and is influenced by the structures of its social environment. In this paper, we provide an overview of key theory and methods in network analysis (especially for social systems) as an introducti…
Vidyasirimedhi Institute of Science and Technology（威达亚西里梅迪科技学院）；Imperial College London（帝国理工学院）；King Mongkut’s University of Technology Thonburi（国王科技大学）
Supervised deep convolutional neural networks (DCNNs) are currently one of the best computational models that can explain how the primate ventral visual stream solves object recognition. However, embodied cognition has not been considered in the existing visual processing models. From the ecological standpoint, humans learn to recognize objects by interacting with them, allowing better classification, specialization, and generalization. Here, we ask if computational models under the embodied learning framework can explain mechanisms underlying object recognition in the primate visual system be…
Deep Neural Networks and Brain Alignment: Brain Encoding and Decoding (Survey)
Inria Bordeaux（法国国家信息与自动化研究所波尔多分所）；LaBRI, Université de Bordeaux（波尔多大学拉布里实验室）；Université de Bordeaux（波尔多大学）；TU Berlin（柏林工业大学）；National University of Singapore（新加坡国立大学）；Microsoft（微软公司）；International Institute of Information Technology Hyderabad（海得拉巴国际信息技术研究所）；CEA（法国原子能和替代能源委员会）；CNRS（法国国家科学研究中心）
Digital twin brain: a bridge between biological intelligence and artificial intelligence
Brainnetome Center, Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所脑网络组研究中心）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；Research Center for Augmented Intelligence, Zhejiang Lab（之江实验室增强智能研究中心）
The language of graph theory, or network science, has proven to be an exceptional tool for addressing myriad problems in neuroscience. Yet, the use of networks is predicated on a critical simplifying assumption: that the quintessential unit of interest in a brain is a dyad -- two nodes (neurons or brain regions) connected by an edge. While rarely mentioned, this fundamental assumption inherently limits the types of neural structure and function that graphs can be used to model. Here, we describe a generalization of graphs that overcomes these limitations, thereby offering a broad range of new …
