Cognitive neuroscience explores the mechanisms of cognition by studying its structural and functional brain correlates. Here, we report the first systematic review that assesses how information from structural and functional neuroimaging methods can be integrated to investigate the brain substrates of cognition. Web of Science and Scopus databases were searched for studies of healthy young adult populations that collected cognitive data, and structural and functional neuroimaging data. Five percent of screened studies met all inclusion criteria. Next, 54% of included studies related cognitive performance to brain structure and function without quantitative analysis of the relationship. Finally, 32% of studies formally integrated structural and functional brain data. Overall, many studies consider either structural or functional neural correlates of cognition, and of those that consider both, they have rarely been integrated. We identified four emergent approaches to the characterisation of the relationship between brain structure, function and cognition; comparative, predictive, fusion and complementary. We discuss the insights provided each approach and how authors can select approaches to suit their research questions.
Cognitive function requires the coordination of neural activity across many scales, from neurons and circuits to large-scale networks. As such, it is unlikely that an explanatory framework focused upon any single scale will yield a comprehensive theory of brain activity and cognitive function. Modelling and analysis methods for neuroscience should aim to accommodate multiscale phenomena. Emerging research now suggests that multi-scale processes in the brain arise from so-called critical phenomena that occur very broadly in the natural world. Criticality arises in complex systems perched betwee…
A pervasive research protocol of cognitive neuroscience is to train subjects to perform deliberately designed experiments and record brain activity simultaneously, aiming to understand the brain mechanism underlying cognition. However, how the results of this protocol can be applied in technology is seldom discussed. Here, I review the studies on time processing of the brain as examples of this protocol, as well as two main application areas of neuroscience (neuroengineering and brain-inspired artificial intelligence). Time processing is an indispensable dimension of cognition; time is also an…
In this paper, I examine what I refer to as the spike doctrine, which is the generally held belief in neuroscience that information in the brain is encoded by sequences of neural action potentials. I present the argument that specific neurochemicals, and not spikes, are the elementary units of information in the brain. I outline several predictions that arise from this interpretation, relate them to results in the current research literature, and show how they address some open questions.
To learn how cognition is implemented in the brain, we must build computational models that can perform cognitive tasks, and test such models with brain and behavioral experiments. Cognitive science has developed computational models of human cognition, decomposing task performance into computational components. However, its algorithms still fall short of human intelligence and are not grounded in neurobiology. Computational neuroscience has investigated how interacting neurons can implement component functions of brain computation. However, it has yet to explain how those components interact …
We describe a mathematical models of grounded symbols in the brain. It also serves as a computational foundations for Perceptual Symbol System (PSS). This development requires new mathematical methods of dynamic logic (DL), which have overcome limitations of classical artificial intelligence and connectionist approaches. The paper discusses these past limitations, relates them to combinatorial complexity (exponential explosion) of algorithms in the past, and further to the static nature of classical logic. The new mathematical theory, DL, is a process-logic. A salient property of this process …
Current theoretical and computational models of dopamine-based reinforcement learning are largely rooted in the classical behaviorist tradition, and envision the organism as a purely reactive recipient of rewards and punishments, with resulting behavior that essentially reflects the sum of this reinforcement history. This framework is missing some fundamental features of the affective nervous system, most importantly, the central role of goals in driving and organizing behavior in a teleological manner. Even when goal-directed behaviors are considered in current frameworks, they are typically …
Physical symbol systems are needed for open-ended cognition. A good way to understand physical symbol systems is by comparison of thought to chemistry. Both have systematicity, productivity and compositionality. The state of the art in cognitive architectures for open-ended cognition is critically assessed. I conclude that a cognitive architecture that evolves symbol structures in the brain is a promising candidate to explain open-ended cognition. Part 2 of the paper presents such a cognitive architecture.
This paper describes the outlines of a research program for understanding the cognitive-emotional brain, with an emphasis on the issue of dynamics: How can we study, characterize, and understand the neural underpinnings of cognitive-emotional behaviors as inherently dynamic processes? The framework embraces many of the central themes developed by Steve Grossberg in his extensive body of work in the past 50 years. By embracing head on the leitmotifs of dynamics, decentralized computation, emergence, selection and competition, and autonomy, it is proposed that a science of the mind-brain can be …
Tracking the positions of objects in local space is a core function of animal brains. We do not yet understand how it is done with limited neural resources. The challenges of spatial cognition are discussed under the criteria: (a) scaling of computational costs; (b) feature binding; (c) precise calculation of spatial displacements; (d) fast learning of invariant patterns; and (e) exploiting the strong Bayesian prior of object constancy. The leading current models of spatial cognition are Hierarchical Bayesian models of vision, and Deep Neural Nets. These are typically fully distributed models,…
We seek general principles of the structure of the cellular collective activity associated with conscious awareness. Can we obtain evidence for features of the optimal brain organization that allows for adequate processing of stimuli and that may guide the emergence of cognition and consciousness? Analysing brain recordings in conscious and unconscious states, we followed initially the classic approach in physics when it comes to understanding collective behaviours of systems composed of a myriad of units: the assessment of the number of possible configurations (microstates) that the system ca…
ETIS Lab, UMR8051, CY Cergy Paris University, ENSEA, CNRS（ETIS实验室，UMR8051，塞尔吉巴黎大学，ENSEA，法国国家科学研究中心）
In order to keep trace of information and grow up, the infant brain has to resolve the problem about where old information is located and how to index new ones. We propose that the immature prefrontal cortex (PFC) use its primary functionality of detecting hierarchical patterns in temporal signals as a second purpose to organize the spatial ordering of the cortical networks in the developing brain itself. Our hypothesis is that the PFC detects the hierarchical structure in temporal sequences in the form of ordinal patterns and use them to index information hierarchically in different parts of …
The cognitive functions of human and non-human primates rely on the dynamic interplay of distributed neural assemblies. As such, it seems unlikely that cognition can be supported by macroscopic brain dynamics at the proximity of thermodynamic equilibrium. We confirmed this hypothesis by investigating electrocorticography data from non human primates undergoing different states of unconsciousness (sleep, and anesthesia with propofol, ketamine, and ketamine plus medetomidine), and funcional magnetic resonance imaging data from humans, both during deep sleep and under propofol anesthesia. Systema…
Synaptic plasticity and neuron cross-talk are some of the important key mechanisms underlying formation of dynamic clusters of active neurons. The essence of this study is to model and decipher the mechanism of emergence of a task-specific functional connectivity among a population of neurons. We have used our proposed neuron activity pattern (NAP) model to define an assorted range of interactions and simulated across a 2D lattice with two-state (firing/non-firing) neurons. We observed the emergence of scale-free, hierarchically organized ordered patterns of active neurons, that we call as fun…
Overlooked weak structural connections support human cognition under nonlinear connectome scaling
Xi’an Jiaotong University（西安交通大学）；Hong Kong Baptist University（香港浸会大学）；Carl von Ossietzky Universität Oldenburg（奥尔登堡卡尔·冯·奥西茨基大学）；University of California San Francisco（加州大学旧金山分校）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；University of Roehampton（罗汉普顿大学）；Zhejiang University（浙江大学）
Humans have been aware for thousands of years that sleep comes in many forms, accompanied by different kinds of mental content. We review the first-person report literature on the frequency and type of content experienced in various stages of sleep, showing that different sleep stages are dissociable at the subjective level. We then relate these subjective differences to the growing literature differentiating the various sleep stages at the neurophysiological level, including evidence from electrophysiology, neurochemistry, and functional neuroimaging. We suggest that there is emerging evidenc…
Diverse cognitive processes set different demands on locally segregated and globally integrated brain activity. However, it remains unclear how resting brains configure their functional organization to balance the demands on network segregation and integration to best serve cognition. Here, we use an eigenmode-based approach to identify hierarchical modules in functional brain networks, and quantify the functional balance between network segregation and integration. In a large sample of healthy young adults (n=991), we combine the whole-brain resting state functional magnetic resonance imaging…
University of Cambridge（剑桥大学）；Intel Labs（英特尔实验室）；Julius-Maximilians-Universität Würzburg（维尔茨堡朱利叶斯-马克西米利安大学）
There is a concerted effort to build domain-general artificial intelligence in the form of universal neural network models with sufficient computational flexibility to solve a wide variety of cognitive tasks but without requiring fine-tuning on individual problem spaces and domains. To do this, models need appropriate priors and inductive biases, such that trained models can generalise to out-of-distribution examples and new problem sets. Here we provide an overview of the hallmarks endowing biological neural networks with the functionality needed for flexible cognition, in order to establish …
It has been said that complexity lies between order and disorder. In the case of brain activity, and physiology in general, complexity issues are being considered with increased emphasis. We sought to identify features of brain organization that are optimal for sensory processing, and that may guide the emergence of cognition and consciousness, by analysing neurophysiological recordings in conscious and unconscious states. We find a surprisingly simple result: normal wakeful states are characterised by the greatest number of possible configurations of interactions between brain networks, repre…
Artificial intelligence has made great strides in the last decade but still falls short of the human brain, the best-known example of intelligence. Not much is known of the neural processes that allow the brain to make the leap to achieve so much from so little beyond its ability to create knowledge structures that can be flexibly and dynamically combined, recombined, and applied in new and novel ways. This paper proposes a mathematical approach using graph theory and spectral graph theory, to hypothesize how to constrain these neural clusters of information based on eigen-relationships. This …
Cognitive neuroscience is enjoying rapid increase in extensive public brain-imaging datasets. It opens the door to large-scale statistical models. Finding a unified perspective for all available data calls for scalable and automated solutions to an old challenge: how to aggregate heterogeneous information on brain function into a universal cognitive system that relates mental operations/cognitive processes/psychological tasks to brain networks? We cast this challenge in a machine-learning approach to predict conditions from statistical brain maps across different studies. For this, we leverage…
Tulane University（杜兰大学）；Central South University（中南大学）；Mind Research Network（心智研究网络）；University of Nebraska Medical Center（内布拉斯加大学医学中心）；Georgia State University（佐治亚州立大学）；Georgia Institute of Technology（佐治亚理工学院）；Emory University（埃默里大学）
Multimodal fusion benefits disease diagnosis by providing a more comprehensive perspective. Developing algorithms is challenging due to data heterogeneity and the complex within- and between-modality associations. Deep-network-based data-fusion models have been developed to capture the complex associations and the performance in diagnosis has been improved accordingly. Moving beyond diagnosis prediction, evaluation of disease mechanisms is critically important for biomedical research. Deep-network-based data-fusion models, however, are difficult to interpret, bringing about difficulties for st…
The vulnerable male brain: Men's spatial abilities are condition-dependent, sexually selected traits
University of Missouri（密苏里大学）；University of Genoa（热那亚大学）；Middlebury College（米德尔伯里学院）；California State University, Fullerton（加州州立大学富勒顿分校）；University of Roehampton（罗汉普顿大学）
The social brain hypothesis postulates the increasing complexity of social interactions as a driving force for the evolution of cognitive abilities. Whereas dyadic and triadic relations play a basic role in defining social behaviours and pose many challenges for the social brain, individuals in animal societies typically belong to relatively large networks. How the structure and dynamics of these networks also contribute to the evolution of cognition, and vice versa, is less understood. Here we review how collective phenomena can occur in systems where social agents do not require sophisticate…
The human brain network is modular--comprised of communities of tightly interconnected nodes. This network contains local hubs, which have many connections within their own communities, and connector hubs, which have connections diversely distributed across communities. A mechanistic understanding of these hubs and how they support cognition has not been demonstrated. Here, we leveraged individual differences in hub connectivity and cognition. We show that a model of hub connectivity accurately predicts the cognitive performance of 476 individuals in four distinct tasks. Moreover, there is a g…
Subcortical Shape Variations and Their Associations with Cognition Across the 8th Decade of Life. A Study in the Lothian Birth Cohort 1936
Department of Neuroimaging Sciences（神经影像科学系）；University of Edinburgh（爱丁堡大学）；Computer Graphics and Visualization Laboratory（计算机图形与可视化实验室）；Korea Advanced Institute of Science and Technology (KAIST)（韩国科学技术院）；Department of Psychology（心理学系）；Edinburgh Futures Institute（爱丁堡未来研究所）
Understanding neurocognitive computations will require not just localizing cognitive information distributed throughout the brain but also determining how that information got there. We review recent advances in linking empirical and simulated brain network organization with cognitive information processing. Building on these advances, we offer a new framework for understanding the role of connectivity in cognition - network coding (encoding/decoding) models. These models utilize connectivity to specify the transfer of information via neural activity flow processes, successfully predicting the…
The machinery of the human brain -- analog, probabilistic, embodied -- can be characterized computationally, but what machinery confers what computational powers? Any such system can be abstractly cast in terms of two computational components: a finite state machine carrying out computational steps, whether via currents, chemistry, or mechanics; plus a set of allowable memory operations, typically formulated in terms of an information store that can be read from and written to, whether via synaptic change, state transition, or recurrent activity. Probing these mechanisms for their information …
Analysis of grip force signals tailored to hand and finger movement evolution and changes in grip force control during task execution provide unprecedented functional insight into somatosensory cognition. Somatosensory cognition is the basis of our ability to act upon and to transform the physical world around us, to recognize objects on the basis of touch alone, and to grasp them with the right amount of force for lifting and manipulating them. Recent technology has permitted the wireless monitoring of grip force signals recorded from biosensors in the palm of the human hand to track and trac…
The possibility that quantum processing with nuclear spins might be operative in the brain is proposed and then explored. Phosphorus is identified as the unique biological element with a nuclear spin that can serve as a qubit for such putative quantum processing - a neural qubit - while the phosphate ion is the only possible qubit-transporter. We identify the "Posner molecule", $\text{Ca}_9 (\text{PO}_4)_6$, as the unique molecule that can protect the neural qubits on very long times and thereby serve as a (working) quantum-memory. A central requirement for quantum-processing is quantum entang…
With the increasing acquisition of large-scale neural recordings comes the challenge of inferring the computations they perform and understanding how these give rise to behavior. Here, we review emerging conceptual and technological advances that begin to address this challenge, garnering insights from both biological and artificial neural networks. We argue that neural data should be recorded during rich behavioral tasks, to model cognitive processes and estimate latent behavioral variables. Careful quantification of animal movements can also provide a more complete picture of how movements s…
Functional connectomes capture brain interactions via synchronized fluctuations in the functional magnetic resonance imaging signal. If measured during rest, they map the intrinsic functional architecture of the brain. With task-driven experiments they represent integration mechanisms between specialized brain areas. Analyzing their variability across subjects and conditions can reveal markers of brain pathologies and mechanisms underlying cognition. Methods of estimating functional connectomes from the imaging signal have undergone rapid developments and the literature is full of diverse stra…
We propose a model of processing of information in the brain which has the following distinguishing features: a). It is quantum-like (QL). The brain uses the quantum rule (given by von Neumann trace formula) for calculation of averages for psychological functions. b). Those functions are considered as self-observations of the brain. c). The QL-representation has the temporal basis. The brain is a machine transferring time into cognition. d). Any cognitive process is based on (at least) two time scales: precognitive time scale (which is very fine) and cognitive time scale (which is essentially …
Rate-Coding Bundle Memory: A Unified Model of Memory and Control for Symbolic Computation in the Brain
Max Planck Institute for Psycholinguistics（马克斯·普朗克心理语言学研究所）；Donders Institute for Brain, Cognition and Behaviour（唐德斯大脑、认知与行为研究所）；Radboud University（拉德堡德大学）
Connectomics and network neuroscience offer quantitative scientific frameworks for modeling and analyzing networks of structurally and functionally interacting neurons, neuronal populations, and macroscopic brain areas. This shift in perspective and emphasis on distributed brain function has provided fundamental insight into the role played by the brain's network architecture in cognition, disease, development, and aging. In this chapter, we review the core concepts of human connectomics at the macroscale. From the construction of networks using functional and diffusion MRI data, to their subs…
Building machines that learn and think like humans is essential not only for cognitive science, but also for computational neuroscience, whose ultimate goal is to understand how cognition is implemented in biological brains. A new cognitive computational neuroscience should build cognitive-level and neural- level models, understand their relationships, and test both types of models with both brain and behavioral data.
The time-changing nature of our world demands processing of huge amounts of information in fast and reliable way to generate successful behaviors. Therefore, significant brain resources are devoted to process spatiotemporal information. Neural basis of spatial processing and their cognitive correlates are well established mostly for static environments. Nonetheless, in time-changing situations the brain exploits specific processing mechanisms for temporal information based on prediction and anticipation, as time compression during visual perception and mental navigation. Alternative hypothesis…
Cognition involves dynamic reconfiguration of functional brain networks at sub-second time scale. A precise tracking of these reconfigurations to categorize visual objects remains elusive. Here, we use dense electroencephalography (EEG) data recorded during naming meaningful (tools, animals) and scrambled objects from 20 healthy subjects. We combine technique for identifying functional brain networks and recently developed algorithm for estimating networks similarity to discriminate between the two categories. First, we showed that dynamic networks of both categories can be segmented into seve…
In cognitive network neuroscience, the connectivity and community structure of the brain network is related to cognition. Much of this research has focused on two measures of connectivity - modularity and flexibility - which frequently have been examined in isolation. By using resting state fMRI data from 52 young adults, we investigate the relationship between modularity, flexibility and performance on cognitive tasks. We show that flexibility and modularity are highly negatively correlated. However, we also demonstrate that flexibility and modularity make unique contributions to explain task…
The focus of the efforts for defining and modelling emotion is broadly shifting from classical definite marker theory to statistically context situated conceptual theory. However, the role of context processing and its interaction with the affect is still not comprehensively explored and modelled. With the help of neural decoding of functional networks, we have decoded cognitive functions for 12 different basic and complex emotion conditions. Using transfer learning in deep neural architecture, we arrived at the conclusion that the core affect is unable to provide varieties of emotions unless …
We adapt an information theory analysis of interacting cognitive biological and social modules to the problem of the global neuronal workspace, the current standard neuroscience picture of consciousness. Tunable punctuation emerges in a natural manner, suggesting the possibility of fitting appropriate phase transition power law, and, away from transition, generalized Onsager relation expressions, to observational data on conscious reaction. The development can be extended in a straightforward way to include the role of psychosocial stress, culture, or other embedding structured contexts in ind…
Virginia Tech（弗吉尼亚理工大学）；The University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）；Duke University（杜克大学）
There has been huge interest in studying human brain connectomes inferred from different imaging modalities and exploring their relationship with human traits, such as cognition. Brain connectomes are usually represented as networks, with nodes corresponding to different regions of interest (ROIs) and edges to connection strengths between ROIs. Due to the high-dimensionality and non-Euclidean nature of networks, it is challenging to depict their population distribution and relate them to human traits. Current approaches focus on summarizing the network using either pre-specified topological fe…
University of Oxford（牛津大学）；Stanford University（斯坦福大学）；University College London（伦敦大学学院）
Many deep neural network architectures loosely based on brain networks have recently been shown to replicate neural firing patterns observed in the brain. One of the most exciting and promising novel architectures, the Transformer neural network, was developed without the brain in mind. In this work, we show that transformers, when equipped with recurrent position encodings, replicate the precisely tuned spatial representations of the hippocampal formation; most notably place and grid cells. Furthermore, we show that this result is no surprise since it is closely related to current hippocampal…
Bayesian models of cognition hypothesize that human brains make sense of data by representing probability distributions and applying Bayes' rule to find the best explanation for available data. Understanding the neural mechanisms underlying probabilistic models remains important because Bayesian models provide a computational framework, rather than specifying mechanistic processes. Here, we propose a deterministic neural-network model which estimates and represents probability distributions from observable events --- a phenomenon related to the concept of probability matching. Our model learns…
The human brain is a complex network of interconnected brain regions organized into functional modules with distinct roles in cognition and behavior. An important question concerns the persistence and stability of these modules over the human lifespan. Here we use graph-theoretic analysis to algorithmically uncover the brain's intrinsic modular organization across multiple spatial scales ranging from small communities comprised of only a few brain regions to large communities made up of many regions. We find that at coarse scales modules become progressively more segregated, while at finer sca…
