The vertebrate hippocampus is believed to use recurrent connectivity in area CA3 to support episodic memory recall from partial cues. This brain area also contains place cells, whose location-selective firing fields implement maps supporting spatial memory. Here we show that place cells emerge in networks trained to remember temporally continuous sensory episodes. We model CA3 as a recurrent autoencoder that recalls and reconstructs sensory experiences from noisy and partially occluded observations by agents traversing simulated rooms. The agents move in realistic trajectories modeled from rodents and environments are modeled as high-dimensional sensory experience maps. Training our autoencoder to pattern-complete and reconstruct experiences with a constraint on total activity causes spatially localized firing fields, i.e., place cells, to emerge in the encoding layer. The emergent place fields reproduce key aspects of hippocampal phenomenology: a) remapping (maintenance of and reversion to distinct learned maps in different environments), implemented via repositioning of experience manifolds in the network's hidden layer, b) orthogonality of spatial representations in different arenas, c) robust place field emergence in differently shaped rooms, with single units showing multiple place fields in large or complex spaces, and d) slow representational drift of place fields. We argue that these results arise because continuous traversal of space makes sensory experience temporally continuous. We make testable predictions: a) rapidly changing sensory context will disrupt place fields, b) place fields will form even if recurrent connections are blocked, but reversion to previously learned representations upon remapping will be abolished, c) the dimension of temporally smooth experience sets the dimensionality of place fields, including during virtual navigation of abstract spaces.
Okinawa Institute of Science and Technology（冲绳科学技术研究所）；National Center of Neurology and Psychiatry（国立神经精神医疗研究中心）；Waseda University（早稻田大学）
The brain attenuates its responses to self-produced exteroceptions (e.g., we cannot tickle ourselves). Is this phenomenon, known as sensory attenuation, enabled innately, or acquired through learning? Here, our simulation study using a multimodal hierarchical recurrent neural network model, based on variational free-energy minimization, shows that a mechanism for sensory attenuation can develop through learning of two distinct types of sensorimotor experience, involving self-produced or externally produced exteroceptions. For each sensorimotor context, a particular free-energy state emerged th…
In sensory neurons the presence of noise can facilitate the detection of weak information-carrying signals, which are encoded and transmitted via correlated sequences of spikes. Here we investigate relative temporal order in spike sequences induced by a subthreshold periodic input, in the presence of white Gaussian noise. To simulate the spikes, we use the FitzHugh-Nagumo model, and to investigate the output sequence of inter-spike intervals (ISIs), we use the symbolic method of ordinal analysis. We find different types of relative temporal order, in the form of preferred ordinal patterns whic…
We study the emergence and coherence of stochastic oscillations in star networks of excitable elements in which peripheral nodes receive independent random inputs. A biophysical model of a distal branch of sensory neuron in which peripheral nodes of Ranvier are coupled to a central node by myelinated cable segments is used along with a generic model of networked stochastic active rotators. We show that coherent oscillations can emerge due to stochastic synchronization of peripheral nodes and that the degree of coherence can be maximized by tuning the coupling strength and the size of the netwo…
The perceptual experience of architecture is enacted by the sensory and motor system. When we act, we change the perceived environment according to a set of expectations that depend on our body and the built environment. The continuous process of collecting sensory information is thus based on bodily affordances. Affordances characterize the fit between the physical structure of the body and capacities for movement in the built environment. Since little has been done regarding the role of architectural design in the emergence of perceptual experience on a neuronal level, this paper offers a fi…
Max-Planck Institute for Mathematics in the Sciences（马克斯·普朗克数学科学研究所）
To date a number of studies have shown that receptive field shapes of early sensory neurons can be reproduced by optimizing coding efficiency of natural stimulus ensembles. A still unresolved question is whether the efficient coding hypothesis explains formation of neurons which explicitly represent environmental features of different functional importance. This paper proposes that the spatial selectivity of higher auditory neurons emerges as a direct consequence of learning efficient codes for natural binaural sounds. Firstly, it is demonstrated that a linear efficient coding transform - Inde…
When brain signals are recorded in an electroencephalogram or some similar large-scale record of brain activity, oscillatory patterns are typically observed that are thought to reflect the aggregate electrical activity of the underlying neuronal ensemble. Although it now seems that such patterns participate in feedback loops both temporally with the neurons' spikes and spatially with other brain regions, the mechanisms that might explain the existence of such loops have remained essentially unknown. Here we present a theoretical study of these issues on a cortical model we introduced earlier […
Exploiting data invariances is crucial for efficient learning in both artificial and biological neural circuits. Understanding how neural networks can discover appropriate representations capable of harnessing the underlying symmetries of their inputs is thus crucial in machine learning and neuroscience. Convolutional neural networks, for example, were designed to exploit translation symmetry and their capabilities triggered the first wave of deep learning successes. However, learning convolutions directly from translation-invariant data with a fully-connected network has so far proven elusive…
University of Surrey（萨里大学）；University of Applied Sciences Stralsund（施特拉尔松德应用技术大学）
Experimental studies support the notion of spike-based neuronal information processing in the brain, with neural circuits exhibiting a wide range of temporally-based coding strategies to rapidly and efficiently represent sensory stimuli. Accordingly, it would be desirable to apply spike-based computation to tackling real-world challenges, and in particular transferring such theory to neuromorphic systems for low-power embedded applications. Motivated by this, we propose a new supervised learning method that can train multilayer spiking neural networks to solve classification problems based on …
Sufficient conditions for offline reactivation in recurrent neural networks
Mila – Quebec AI Institute（魁北克AI研究所）；Université de Montréal（蒙特利尔大学）；McGill University（麦吉尔大学）；Canada CIFAR AI Chair（加拿大CIFAR人工智能 chair）；CIFAR Learning in Machines & Brains（CIFAR机器与大脑学习）
Mechanisms underlying the emergence of orientation selectivity in the primary visual cortex are highly debated. Here we study the contribution of inhibition-dominated random recurrent networks to orientation selectivity, and more generally to sensory processing. By simulating and analyzing large-scale networks of spiking neurons, we investigate tuning amplification and contrast invariance of orientation selectivity in these networks. In particular, we show how selective attenuation of the common mode and amplification of the modulation component take place in these networks. Selective attenuat…
Neuronal circuits can learn and replay firing patterns evoked by sequences of sensory stimuli. After training, a brief cue can trigger a spatiotemporal pattern of neural activity similar to that evoked by a learned stimulus sequence. Network models show that such sequence learning can occur through the shaping of feedforward excitatory connectivity via long term plasticity. Previous models describe how event order can be learned, but they typically do not explain how precise timing can be recalled. We propose a mechanism for learning both the order and precise timing of event sequences. In our…
Fascinating and puzzling phenomena, such as landmark vector cells, splitter cells, and event-specific representations to name a few, are regularly discovered in the hippocampus. Without a unifying principle that can explain these divergent observations, each experiment seemingly discovers a new anomaly or coding type. Here, we provide a unifying principle that the mental representation of space is an emergent property of latent higher-order sequence learning. Treating space as a sequence resolves myriad phenomena, and suggests that the place-field mapping methodology where sequential neuron re…
