An increasing body of evidence suggests that the trial-to-trial variability of spiking activity in the brain is not mere noise, but rather the reflection of a sampling-based encoding scheme for probabilistic computing. Since the precise statistical properties of neural activity are important in this context, many models assume an ad-hoc source of well-behaved, explicit noise, either on the input or on the output side of single neuron dynamics, most often assuming an independent Poisson process in either case. However, these assumptions are somewhat problematic: neighboring neurons tend to share receptive fields, rendering both their input and their output correlated; at the same time, neurons are known to behave largely deterministically, as a function of their membrane potential and conductance. We suggest that spiking neural networks may, in fact, have no need for noise to perform sampling-based Bayesian inference. We study analytically the effect of auto- and cross-correlations in functionally Bayesian spiking networks and demonstrate how their effect translates to synaptic interaction strengths, rendering them controllable through synaptic plasticity. This allows even small ensembles of interconnected deterministic spiking networks to simultaneously and co-dependently shape their output activity through learning, enabling them to perform complex Bayesian computation without any need for noise, which we demonstrate in silico, both in classical simulation and in neuromorphic emulation. These results close a gap between the abstract models and the biology of functionally Bayesian spiking networks, effectively reducing the architectural constraints imposed on physical neural substrates required to perform probabilistic computing, be they biological or artificial.
Our current understanding of brain rhythms is based on quantifying their instantaneous or time-averaged characteristics. What remains unexplored, is the actual structure of the waves -- their shapes and patterns over finite timescales. To address this, we used two independent approaches to link wave forms to their physiological functions: the first is based on quantifying their consistency with the underlying mean behavior, and the second assesses "orderliness" of the waves' features. The corresponding measures capture the wave's characteristic and abnormal behaviors, such as atypical periodic…
What can we hope to learn about brains from the free energy principle? In adopting the "primordial soup" physical model, Bruineberg et al. perpetuate the unsupported notion that the free energy principle has a meaningful physical--and neuronal--interpretation. We examine how minimization of free energy arises in physical contexts, and what this can and cannot tell us about brains.
Brain stimulation is a core method in neuroscience. Numerous non-invasive brain stimulation (NIBS) techniques are currently in use in basic and clinical research, and recent advances promise the ability to non-invasively access deep brain structures. While encouraging, there is a surprising gap in our understanding of precisely how NIBS perturbs neural activity throughout an interconnected network, and how such perturbed neural activity ultimately links to behaviour. In this review, we will consider why non-human primate (NHP) models of NIBS are ideally situated to address this gap in knowledg…
Selection through iterated learning explains no more than other non-functional accounts, such as universal grammar, why language is so well-designed for communicative efficiency. It does not predict several distinctive features of language like central embedding, large lexicons or the lack of iconicity, that seem to serve communication purposes at the expense of learnability.
The relationship between brains and computers is often taken to be merely metaphorical. However, genuine computational systems can be implemented in virtually any media; thus, one can take seriously the view that brains literally compute. But without empirical criteria for what makes a physical system genuinely a computational one, computation remains a matter of perspective, especially for natural systems (e.g., brains) that were not explicitly designed and engineered to be computers. Considerations from real examples of physical computers-both analog and digital, contemporary and historical-…
Scientific studies have shown that non-conscious stimuli and representations influence information processing during conscious experience. In the light of such evidence, questions about potential functional links between non-conscious brain representations and conscious experience arise. This article discusses a neural coding model capable of explaining how statistical learning mechanisms in dedicated resonant circuits could generate specific temporal activity traces of non-conscious representations in the brain. How reentrant signaling, top-down matching, and statistical coincidence of such a…
An heuristic signal processing scheme recently introduced shows how brain signals can be efficiently represented by a sparse spatiotemporal point process. The approach has been validated already for different relevant conditions demonstrating that preserves and compress a surprisingly large fraction of the signal information. In this paper the conditions for such compression to succeed are investigated as well as the underlying reasons for such good performance. The results show that the key lies in the correlation properties of the time series under consideration. It is found that signals wit…
As the field of connectomics has matured, it has expanded from mapping the existence of connections between brain components to measuring the strength of connections. This information is increasingly accessible via methodologies such as pairing functional magnetic resonance (MR) imaging and MR tractography in the same human subject, as well as novel methods in non-human animals using optogenetics. Systems and network neuroscience have in recent years focused extensively on explaining correlation patterns of functional activity in the brain in terms of the degree of connectedness of brain compo…
The ubiquity of topographic maps in the brain has long been known, and molecular mechanisms for the formation of topographic organization of neural systems have been revealed. Less attention has been given to the question of why are the maps topographical and why so ubiquitous. In this study, I explore the implications of the topographic maps for brain function, by employing the mathematical framework of the Zeeman topology. I propose the notion about the meaning of topographic order as generic mechanisms for the representation and analysis of causal structure, implemented by the neural system…
Why the Brain Consolidates: Predictive Forgetting for Optimal Generalisation
Huawei Noah’s Ark Lab（华为诺亚实验室）；AI Centre, Department of Computer Science, University College London（大学计算机科学系人工智能中心）；UCL Institute of Cognitive Neuroscience, University College London（大学认知神经科学研究所）；UCL Queen Square Institute of Neurology, University College London（大学Queen Square神经学研究所）
A key element of the European Union's Human Brain Project (HBP) and other large-scale brain research projects is simulation of large-scale model networks of neurons. Here we argue why such simulations will likely be indispensable for bridging the scales between the neuron and system levels in the brain, and a set of brain simulators based on neuron models at different levels of biological detail should thus be developed. To allow for systematic refinement of candidate network models by comparison with experiments, the simulations should be multimodal in the sense that they should not only pred…
In recent years, several studies have provided insight on the functioning of the brain which consists of neurons and form networks via interconnection among them by synapses. Neural networks are formed by interconnected systems of neurons, and are of two types, namely, the Artificial Neural Network (ANNs) and Biological Neural Network (interconnected nerve cells). The ANNs are computationally influenced by human neurons and are used in modelling neural systems. The reasoning foundations of ANNs have been useful in anomaly detection, in areas of medicine such as instant physician, electronic no…
Targeted stimulation of the brain has the potential to treat mental illnesses. We propose an approach to help design the stimulation protocol by identifying electrical dynamics across many brain regions that relate to illness states. We model multi-region electrical activity as a superposition of activity from latent networks, where the weights on the latent networks relate to an outcome of interest. In order to improve on drawbacks of latent factor modeling in this context, we focus on supervised autoencoders (SAEs), which can improve predictive performance while maintaining a generative mode…
Stroke is one of the main causes of human disabilities. Experimental observations indicate that several mechanisms are activated during the recovery of functional activity after a stroke. Here we unveil how the brain recovers by explaining the role played by three mechanisms: Plastic adaptation, hyperexcitability and synaptogenesis. We consider two different damages in a neural network: A diffuse damage that simply causes the reduction of the effective system size and a localized damage, a stroke, that strongly alters the spontaneous activity of the system. Recovery mechanisms observed experim…
Enormous questions still loom for the emerging science of spontaneous thought: what, exactly, is spontaneous thought? Why does our brain engage in spontaneous forms of thinking, and when is this most likely to occur? And perhaps the question most interesting and accessible from a scientific perspective: how does the brain generate, elaborate, and evaluate its own spontaneous creations? The central aim of this volume is to bring together views from neuroscience, psychology, philosophy, phenomenology, history, education, contemplative traditions, and clinical practice in order to begin to addres…
The accurate diagnosis and assessment of neurodegenerative disease and traumatic brain injuries (TBI) remain open challenges. Both cause cognitive and functional deficits due to focal axonal swellings (FAS), but it is difficult to deliver a prognosis due to our limited ability to assess damaged neurons at a cellular level in vivo. We simulate the effects of neurodegenerative disease and TBI using convolutional neural networks (CNNs) as our model of cognition. We utilize biophysically relevant statistical data on FAS to damage the connections in CNNs in a functionally relevant way. We incorpora…
The human brain exhibits a relatively stable spatiotemporal organization that supports brain function and can be manipulated via local brain stimulation. Such perturbations to local cortical dynamics are globally integrated by distinct neural systems. However, it remains unclear how and why local changes in neural activity affect large-scale system dynamics. Here, we briefly review empirical and computational studies addressing how localized perturbations affect brain activity. We then systematically analyze a model of large-scale brain dynamics, assessing how localized changes in brain activi…
Self-localization during navigation with noisy sensors in an ambiguous world is computationally challenging, yet animals and humans excel at it. In robotics, Simultaneous Location and Mapping (SLAM) algorithms solve this problem though joint sequential probabilistic inference of their own coordinates and those of external spatial landmarks. We generate the first neural solution to the SLAM problem by training recurrent LSTM networks to perform a set of hard 2D navigation tasks that include generalization to completely novel trajectories and environments. The hidden unit representations exhibit…
Computational modeling plays an increasingly important role in neuroscience, highlighting the philosophical question of how computational models explain. In the context of neural network models for neuroscience, concerns have been raised about model intelligibility, and how they relate (if at all) to what is found in the brain. We claim that what makes a system intelligible is an understanding of the dependencies between its behavior and the factors that are causally responsible for that behavior. In biological systems, many of these dependencies are naturally "top-down": ethological imperativ…
Despite the recent success of neural network models in mimicking animal performance on visual perceptual tasks, critics worry that these models fail to illuminate brain function. We take it that a central approach to explanation in systems neuroscience is that of mechanistic modeling, where understanding the system is taken to require fleshing out the parts, organization, and activities of a system, and how those give rise to behaviors of interest. However, it remains somewhat controversial what it means for a model to describe a mechanism, and whether neural network models qualify as explanat…
Open problems abound in the theory of complex networks, which has found successful application to diverse fields of science. With the aim of further advancing the understanding of the brain's functional connectivity, we propose to evaluate a network metric which we term the geodesic entropy. This entropy, in a way that can be made precise, quantifies the Shannon entropy of the distance distribution to a specific node from all other nodes. Measurements of geodesic entropy allow for the characterization of the structural information of a network that takes into account the distinct role of each …
It is proposed that both human creativity and human consciousness are (unintended) consequences of the human brain's extraordinary energy efficiency. The topics of creativity and consciousness are treated separately, though have a common sub-structure. It is argued that creativity arises from a synergy between two cognitive modes of the human brain (which broadly coincide with Kahneman's Systems 1 and 2). In the first, available energy is spread across a relatively large network of neurons. As such, the amount of energy per active neuron is so small that the operation of such neurons is suscep…
As the limits of traditional von Neumann computing come into view, the brain's ability to communicate vast quantities of information using low-power spikes has become an increasing source of inspiration for alternative architectures. Key to the success of these largescale neural networks is a power-efficient spiking element that is scalable and easily interfaced with traditional control electronics. In this work, we present a spiking element fabricated from superconducting nanowires that has pulse energies on the order of ~10 aJ. We demonstrate that the device reproduces essential characterist…
Currently, many studies of Alzheimer's disease (AD) are investigating the neurobiological factors behind the acquisition of beta-amyloid (A), pathologic tau (T), and neurodegeneration ([N]) biomarkers from neuroimages. However, a system-level mechanism of how these neuropathological burdens promote neurodegeneration and why AD exhibits characteristic progression is largely elusive. In this study, we combined the power of systems biology and network neuroscience to understand the dynamic interaction and diffusion process of AT[N] biomarkers from an unprecedented amount of longitudinal Amyloid P…
University College London（伦敦大学学院）；Swiss AI Lab IDSIA（瑞士人工智能实验室IDSIA）；Imperial College London（帝国理工学院）
Under the Bayesian brain hypothesis, behavioural variations can be attributed to different priors over generative model parameters. This provides a formal explanation for why individuals exhibit inconsistent behavioural preferences when confronted with similar choices. For example, greedy preferences are a consequence of confident (or precise) beliefs over certain outcomes. Here, we offer an alternative account of behavioural variability using Rényi divergences and their associated variational bounds. Rényi bounds are analogous to the variational free energy (or evidence lower bound) and can b…
Correlations are employed in modern physics to explain microscopic and macroscopic phenomena, like the fractional quantum Hall effect and the Mott insulator state in high temperature superconductors and ultracold atoms. Simultaneously probed neurons in the intact brain reveal correlations between their activity, an important measure to study information processing in the brain that also influences macroscopic signals of neural activity, like the electro encephalogram (EEG). Networks of spiking neurons differ from most physical systems: The interaction between elements is directed, time delayed…
Efficient coding under constraint drives neural systems towards criticality and sloppiness
Institute of Theoretical Physics, Chinese Academy of Sciences（中国科学院理论物理研究所）；School of Physical Sciences, University of Chinese Academy of Sciences（中国科学院大学物理科学学院）
Natural animal behavior displays rich lexical and temporal dynamics, even in a stable environment. This implies that behavioral variability arises from sources within the brain, but the origin and mechanics of these processes remain largely unknown. Here, we focus on the observation that the timing of self-initiated actions shows large variability even when they are executed in stable, well-learned sequences. Could this mix of reliability and stochasticity arise within the same circuit? We trained rats to perform a stereotyped sequence of self-initiated actions and recorded neural ensemble act…
Neural-network models of high-level brain functions such as memory recall and reasoning often rely on the presence of stochasticity. The majority of these models assumes that each neuron in the functional network is equipped with its own private source of randomness, often in the form of uncorrelated external noise. However, both in vivo and in silico, the number of noise sources is limited due to space and bandwidth constraints. Hence, neurons in large networks usually need to share noise sources. Here, we show that the resulting shared-noise correlations can significantly impair the performa…
How does the size of a neural circuit influence its learning performance? Intuitively, we expect the learning capacity of a neural circuit to grow with the number of neurons and synapses. Larger brains tend to be found in species with higher cognitive function and learning ability. Similarly, adding connections and units to artificial neural networks can allow them to solve more complex tasks. However, we show that in a biologically relevant setting where synapses introduce an unavoidable amount of noise, there is an optimal size of network for a given task. Beneath this optimal size, our anal…
