Although deep RL models have shown a great potential for solving various types of tasks with minimal supervision, several key challenges remain in terms of learning from limited experience, adapting to environmental changes, and generalizing learning from a single task. Recent evidence in decision neuroscience has shown that the human brain has an innate capacity to resolve these issues, leading to optimism regarding the development of neuroscience-inspired solutions toward sample-efficient, and generalizable RL algorithms. We show that the computational model combining model-based and model-free control, which we term the prefrontal RL, reliably encodes the information of high-level policy that humans learned, and this model can generalize the learned policy to a wide range of tasks. First, we trained the prefrontal RL, and deep RL algorithms on 82 subjects' data, collected while human participants were performing two-stage Markov decision tasks, in which we manipulated the goal, state-transition uncertainty and state-space complexity. In the reliability test, which includes the latent behavior profile and the parameter recoverability test, we showed that the prefrontal RL reliably learned the latent policies of the humans, while all the other models failed. Second, to test the ability to generalize what these models learned from the original task, we situated them in the context of environmental volatility. Specifically, we ran large-scale simulations with 10 Markov decision tasks, in which latent context variables change over time. Our information-theoretic analysis showed that the prefrontal RL showed the highest level of adaptability and episodic encoding efficacy. This is the first attempt to formally test the possibility that computational models mimicking the way the brain solves general problems can lead to practical solutions to key challenges in machine learning.
We have developed a model for online continual or lifelong reinforcement learning (RL) inspired on the insect brain. Our model leverages the offline training of a feature extraction and a common general policy layer to enable the convergence of RL algorithms in online settings. Sharing a common policy layer across tasks leads to positive backward transfer, where the agent continuously improved in older tasks sharing the same underlying general policy. Biologically inspired restrictions to the agent's network are key for the convergence of RL algorithms. This provides a pathway towards efficien…
Tongji University（同济大学）；Fudan University（复旦大学）；Technical University of Munich（慕尼黑工业大学）
An effective way to achieve intelligence is to simulate various intelligent behaviors in the human brain. In recent years, bio-inspired learning methods have emerged, and they are different from the classical mathematical programming principle. In the perspective of brain inspiration, reinforcement learning has gained additional interest in solving decision-making tasks as increasing neuroscientific research demonstrates that significant links exist between reinforcement learning and specific neural substrates. Because of the tremendous research that focuses on human brains and reinforcement l…
Sample efficiency has been a key issue in reinforcement learning (RL). An efficient agent must be able to leverage its prior experiences to quickly adapt to similar, but new tasks and situations. Meta-RL is one attempt at formalizing and addressing this issue. Inspired by recent progress in meta-RL, we introduce BIMRL, a novel multi-layer architecture along with a novel brain-inspired memory module that will help agents quickly adapt to new tasks within a few episodes. We also utilize this memory module to design a novel intrinsic reward that will guide the agent's exploration. Our architectur…
Kirchhoff Institute for Physics, Heidelberg University（海德堡大学基尔霍夫物理研究所）；Department of Physiology, University of Bern（伯尔尼大学生理学系）；University of Bern（伯尔尼大学）；Heidelberg University（海德堡大学）；Kirchhoff Institute for Physics（基尔霍夫物理研究所）
Future developments in artificial intelligence will profit from the existence of novel, non-traditional substrates for brain-inspired computing. Neuromorphic computers aim to provide such a substrate that reproduces the brain's capabilities in terms of adaptive, low-power information processing. We present results from a prototype chip of the BrainScaleS-2 mixed-signal neuromorphic system that adopts a physical-model approach with a 1000-fold acceleration of spiking neural network dynamics relative to biological real time. Using the embedded plasticity processor, we both simulate the Pong arca…
Developments in reinforcement learning (RL) have allowed algorithms to achieve impressive performance in highly complex, but largely static problems. In contrast, biological learning seems to value efficiency of adaptation to a constantly-changing world. Here we build on a recently-proposed neuronal learning rule that assumes each neuron can optimize its energy balance by predicting its own future activity. That assumption leads to a neuronal learning rule that uses presynaptic input to modulate prediction error. We argue that an analogous RL rule would use action probability to modulate rewar…
McGovern Brain Imaging Institute, State Key Laboratory of Cognitive Neuroscience and Learning, Beijing Normal University（北京师范大学认知神经科学与学习国家重点实验室麦戈文脑成像研究所）；Tsinghua University（清华大学）；Nanjing University of Aeronautics and Astronautics（南京航空航天大学）；Georgia Institute of Technology（佐治亚理工学院）；Emory University（埃默里大学）；Georgia State University（佐治亚州立大学）；Beijing Normal University（北京师范大学）
Brain-SAD: A Brain-Inspired Safe Autonomous Driving Control Framework with Dynamic Fear-Oriented Constraint on Dual-Policy
Nanjing University of Information Science & Technology（南京信息工程大学）；University of Electronic Science and Technology of China（电子科技大学）；Jiangsu Ocean University（江苏海洋大学）
This article surveys engineering and neuroscientific models of planning as a cognitive function, which is regarded as a typical function of fluid intelligence in the discussion of general intelligence. It aims to present existing planning models as references for realizing the planning function in brain-inspired AI or artificial general intelligence (AGI). It also proposes themes for the research and development of brain-inspired AI from the viewpoint of tasks and architecture.
Artificial Neural Networks (ANNs) were devised as a tool for Artificial Intelligence design implementations. However, it was soon became obvious that they are unable to fulfill their duties. The fully autonomous way of ANNs working, precluded from any human intervention or supervision, deprived of any theoretical underpinning, leads to a strange state of affairs, when ANN designers cannot explain why and how they achieve their amazing and remarkable results. Therefore, contemporary Artificial Intelligence R&D looks more like a Modern Alchemy enterprise rather than a respected scientific or tec…
A method for the ethical analysis of brain-inspired AI
Uppsala University（乌普萨拉大学）；Biogem, Biology and Molecular Genetics Research Institute（比奥盖姆生物与分子遗传学研究所）；National Research Council of Italy（意大利国家研究委员会）；De Montfort University（德蒙福特大学）；University of Bern（伯尔尼大学）；Institute of Neuroethics（神经伦理学研究所）；University of Nottingham（诺丁汉大学）；Jülich Research Centre（于利希研究中心）；University of Cologne（科隆大学）
A differentiable brain simulator bridging brain simulation and brain-inspired computing
School of Software Engineering, Beijing Jiaotong University（北京交通大学软件学院）；Institute of Software, Chinese Academy of Sciences（中国科学院软件研究所）；Academy for Advanced Interdisciplinary Studies, Peking University（北京大学前沿交叉学科研究院）
Brain-inspired Computing Based on Deep Learning for Human-computer Interaction: A Review
Shenyang Institute of Computing Technology, Chinese Academy of Sciences（中国科学院沈阳计算技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Heilongjiang Academy of Sciences Intelligent Manufacturing Institute（黑龙江科学院智能制造研究所）
Neural Structural Reasoner: A Brain-inspired Architecture for Reasoning over Structured Knowledge
Chinese Institute for Brain Research, Beijing（北京脑科学与类脑研究中心）；Beijing Key Laboratory of Brain Science and Brain-Machine Interface（北京脑科学与脑机接口重点实验室）；Sun Yat-sen University（中山大学）
University of Aberdeen（阿伯丁大学）；King’s College（国王学院）
Autonomous driving has attracted great attention from both academics and industries. To realise autonomous driving, Deep Imitation Learning (DIL) is treated as one of the most promising solutions, because it improves autonomous driving systems by automatically learning a complex mapping from human driving data, compared to manually designing the driving policy. However, existing DIL methods cannot generalise well across domains, that is, a network trained on the data of source domain gives rise to poor generalisation on the data of target domain. In the present study, we propose a novel brain-…
BrainNet-MoE: Brain-Inspired Mixture-of-Experts Learning for Neurological Disease Identification
The University of Texas at Arlington（德克萨斯大学阿灵顿分校）；Indiana University Indianapolis（印第安纳大学印第安纳波利斯分校）；University of Cambridge（剑桥大学）；University of Sheffield（谢菲尔德大学）；The University of Georgia（佐治亚大学）
Orthogonal Research and Education Laboratory（正交研究与教育实验室）；OpenWorm Foundation（开放蠕虫基金会）
Artificial Intelligence (AI) systems based solely on neural networks or symbolic computation present a representational complexity challenge. While minimal representations can produce behavioral outputs like locomotion or simple decision-making, more elaborate internal representations might offer a richer variety of behaviors. We propose that these issues can be addressed with a computational approach we call meta-brain models. Meta-brain models are embodied hybrid models that include layered components featuring varying degrees of representational complexity. We will propose combinations of l…
New computing technologies inspired by the brain promise fundamentally different ways to process information with extreme energy efficiency and the ability to handle the avalanche of unstructured and noisy data that we are generating at an ever-increasing rate. To realise this promise requires a brave and coordinated plan to bring together disparate research communities and to provide them with the funding, focus and support needed. We have done this in the past with digital technologies; we are in the process of doing it with quantum technologies; can we now do it for brain-inspired computing…
The University of Georgia（佐治亚大学）；The University of Texas at Arlington（德克萨斯大学阿灵顿分校）；University of Electronic Science and Technology of China（电子科技大学）；Northwestern Polytechnical University（西北工业大学）；Massachusetts General Hospital（麻省总医院）；Harvard Medical School（哈佛医学院）；ShanghaiTech University（上海科技大学）；Shanghai United Imaging Intelligence Co., Ltd.（上海联影智能科技有限公司）
Artificial General Intelligence (AGI) has been a long-standing goal of humanity, with the aim of creating machines capable of performing any intellectual task that humans can do. To achieve this, AGI researchers draw inspiration from the human brain and seek to replicate its principles in intelligent machines. Brain-inspired artificial intelligence is a field that has emerged from this endeavor, combining insights from neuroscience, psychology, and computer science to develop more efficient and powerful AI systems. In this article, we provide a comprehensive overview of brain-inspired AI from …
University of Padova（帕多瓦大学）；Jahangirnagar University（贾汉吉尔纳加尔大学）；American International University - Bangladesh（孟加拉美国国际大学）；University of Sheffield（谢菲尔德大学）；Anglia Ruskin University（安格利亚鲁斯金大学）；Saitama University（埼玉大学）；University of Stirling（斯特灵大学）
Rapid popularity of Internet of Things (IoT) and cloud computing permits neuroscientists to collect multilevel and multichannel brain data to better understand brain functions, diagnose diseases, and devise treatments. To ensure secure and reliable data communication between end-to-end (E2E) devices supported by current IoT and cloud infrastructure, trust management is needed at the IoT and user ends. This paper introduces a Neuro-Fuzzy based Brain-inspired trust management model (TMM) to secure IoT devices and relay nodes, and to ensure data reliability. The proposed TMM utilizes node behavio…
Ritsumeikan University（立命馆大学）；The Whole Brain Architecture Initiative（全脑架构倡议组织）；The University of Tokyo（东京大学）；RIKEN（理化学研究所）
The human brain, among its several functions, analyzes the double articulation structure in spoken language, i.e., double articulation analysis (DAA). A hierarchical structure in which words are connected to form a sentence and words are composed of phonemes or syllables is called a double articulation structure. Where and how DAA is performed in the human brain has not been established, although some insights have been obtained. In addition, existing computational models based on a probabilistic generative model (PGM) do not incorporate neuroscientific findings, and their consistency with the…
Key Laboratory for Radio Astronomy, Chinese Academy of Sciences（中国科学院射电天文重点实验室）；University of Chinese Academy of Sciences（中国科学院大学）
Evolution of deep learning shows that some algorithmic tricks are more durable , while others are not. To the best of our knowledge, we firstly summarize 5 more durable and complete deep learning components for vision, that is, WARSHIP. Moreover, we give a biological overview of WARSHIP, emphasizing brain-inspired computing of WARSHIP. As a step towards WARSHIP, our case study of image super resolution combines 3 components of RSH to deploy a CNN model of WARSHIP-XZNet, which performs a happy medium between speed and performance.
Brain-inspired learning in artificial neural networks: a review
Johns Hopkins University（约翰斯·霍普金斯大学）；University of Cambridge（剑桥大学）；Intel Labs（英特尔实验室）；ML Collective（机器学习联盟）；The Swiss AI Lab IDSIA（瑞士人工智能实验室IDSIA）；University of Maryland, College Park（马里兰大学帕克分校）；University of California, Santa Cruz（加州大学圣克鲁兹分校）
Brain-inspired Evolutionary Architectures for Spiking Neural Networks
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences（中国科学院脑科学与智能技术卓越创新中心）
Memorial Sloan Kettering Cancer Center（纪念斯隆·凯特琳癌症中心）；Indian Institute of Technology Madras（印度理工学院马德拉斯分校）
Artificial intelligence (AI) in radiology has made great strides in recent years, but many hurdles remain. Overfitting and lack of generalizability represent important ongoing challenges hindering accurate and dependable clinical deployment. If AI algorithms can avoid overfitting and achieve true generalizability, they can go from the research realm to the forefront of clinical work. Recently, small data AI approaches such as deep neuroevolution (DNE) have avoided overfitting small training sets. We seek to address both overfitting and generalizability by applying DNE to a virtually pooled dat…
Tsinghua University（清华大学）；University of California, Santa Barbara（加州大学圣巴巴拉分校）
Two main routes of learning methods exist at present including error-driven global learning and neuroscience-oriented local learning. Integrating them into one network may provide complementary learning capabilities for versatile learning scenarios. At the same time, neuromorphic computing holds great promise, but still needs plenty of useful algorithms and algorithm-hardware co-designs for exploiting the advantages. Here, we report a neuromorphic hybrid learning model by introducing a brain-inspired meta-learning paradigm and a differentiable spiking model incorporating neuronal dynamics and …
Continual Learning of Multiple Cognitive Functions with Brain-inspired Temporal Development Mechanism
State Key Laboratory of Brain Cognition and Brain-inspired Intelligence Technology, Chinese Academy of Sciences（中国科学院脑认知与类脑智能技术国家重点实验室）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；Center for Long-term Artificial Intelligence（长期人工智能中心）
Rochester Institute of Technology（罗切斯特理工学院）；Carlson Center for Imaging Science（卡尔森成像科学中心）
Incremental class learning involves sequentially learning classes in bursts of examples from the same class. This violates the assumptions that underlie methods for training standard deep neural networks, and will cause them to suffer from catastrophic forgetting. Arguably, the best method for incremental class learning is iCaRL, but it requires storing training examples for each class, making it challenging to scale. Here, we propose FearNet for incremental class learning. FearNet is a generative model that does not store previous examples, making it memory efficient. FearNet uses a brain-ins…
State Key Laboratory of Media Convergence and Communication, Communication University of China（媒体融合与传播国家重点实验室，中国传媒大学）；Neuroscience and Intelligent Media Institute, Communication University of China（神经科学与智能媒体研究院，中国传媒大学）
The partially occluded image recognition (POIR) problem has been a challenge for artificial intelligence for a long time. A common strategy to handle the POIR problem is using the non-occluded features for classification. Unfortunately, this strategy will lose effectiveness when the image is severely occluded, since the visible parts can only provide limited information. Several studies in neuroscience reveal that feature restoration which fills in the occluded information and is called amodal completion is essential for human brains to recognize partially occluded images. However, feature res…
The Whole Brain Architecture Initiative（全脑架构倡议组织）；The University of Tokyo（东京大学）；RIKEN（理化学研究所）
The vastness of the design space created by the combination of a large number of computational mechanisms, including machine learning, is an obstacle to creating an artificial general intelligence (AGI). Brain-inspired AGI development, in other words, cutting down the design space to look more like a biological brain, which is an existing model of a general intelligence, is a promising plan for solving this problem. However, it is difficult for an individual to design a software program that corresponds to the entire brain because the neuroscientific data required to understand the architectur…
The catastrophic forgetting of previously learnt classes is one of the main obstacles to the successful development of a reliable and accurate generative continual learning model. When learning new classes, the internal representation of previously learnt ones can often be overwritten, resulting in the model's "memory" of earlier classes being lost over time. Recent developments in neuroscience have uncovered a method through which the brain avoids its own form of memory interference. Applying a targeted exaggeration of the differences between features of similar, yet competing memories, the b…
Brain-Inspired Exploration of Functional Networks and Key Neurons in Large Language Models
School of Automation, Northwestern Polytechnical University, Xi’an, China（自动化学院，西北工业大学，西安，中国）；School of Computing, University of Georgia, Athens, USA（计算机学院，佐治亚大学，亚特兰大，美国）；School of Physics and Information Technology, Shaanxi Normal University, Xi’an, China（物理与信息技术学院，陕西师范大学，西安，中国）
A human does not have to see all elephants to recognize an animal as an elephant. On contrast, current state-of-the-art deep learning approaches heavily depend on the variety of training samples and the capacity of the network. In practice, the size of network is always limited and it is impossible to access all the data samples. Under this circumstance, deep learning models are extremely fragile to human-imperceivable adversarial examples, which impose threats to all safety critical systems. Inspired by the association and attention mechanisms of the human brain, we propose reverse adversaria…
Ensemble learning is a classical learning method utilizing a group of weak learners to form a strong learner, which aims to increase the accuracy of the model. Recently, brain-inspired hyperdimensional computing (HDC) becomes an emerging computational paradigm that has achieved success in various domains such as human activity recognition, voice recognition, and bio-medical signal classification. HDC mimics the brain cognition and leverages high-dimensional vectors (e.g., 10000 dimensions) with fully distributed holographic representation and (pseudo-)randomness. This paper presents the first …
