Under the Hood of Neural Networks: Characterizing Learned Representations by Functional Neuron Populations and Network Ablations
Institute of Technologies and Management of Digital Transformation（数字化转型技术与管理研究所）；Institute of Information Management in Mechanical Engineering（机械工程信息管理研究所）
Richard Meyes, Constantin Waubert de Puiseau, Andres Posada-Moreno, Tobias Meisen
The need for more transparency of the decision-making processes in artificial neural networks steadily increases driven by their applications in safety critical and ethically challenging domains such as autonomous driving or medical diagnostics. We address today's lack of transparency of neural networks and shed light on the roles of single neurons and groups of neurons within the network fulfilling a learned task. Inspired by research in the field of neuroscience, we characterize the learned representations by activation patterns and network ablations, revealing functional neuron populations that a) act jointly in response to specific stimuli or b) have similar impact on the network's performance after being ablated. We find that neither a neuron's magnitude or selectivity of activation, nor its impact on network performance are sufficient stand-alone indicators for its importance for the overall task. We argue that such indicators are essential for future advances in transfer learning and modern neuroscience.
Universal Neurons in GPT-2: Emergence, Persistence, and Functional Impact
University of Waterloo（滑铁卢大学）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Ottawa University（渥太华大学）；Algoverse（Algoverse公司）；Meta FAIR Lab（Meta FAIR实验室）；UCSD（加州大学圣地亚哥分校）
Multi-compartment Neuron and Population Encoding Powered Spiking Neural Network for Deep Distributional Reinforcement Learning
Brain-inspired Cognitive Intelligence Lab, Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所类脑认知智能实验室）；Center for Long-term Artificial Intelligence（长期人工智能中心）；University of Chinese Academy of Sciences（中国科学院大学）；Key Laboratory of Brain Cognition and Brain-inspired Intelligence Technology, Chinese Academy of Sciences（中国科学院脑认知与脑智能技术重点实验室）
Learning algorithms are being increasingly adopted in various applications. However, further expansion will require methods that work more automatically. To enable this level of automation, a more powerful solution representation is needed. However, by increasing the representation complexity a second problem arises. The search space becomes huge and therefore an associated scalable and efficient searching algorithm is also required. To solve both problems, first a powerful representation is proposed that unifies most of the neural networks features from the literature into one representation.…
University of Massachusetts Amherst（马萨诸塞大学阿默斯特分校）；Massachusetts Institute of Technology（麻省理工学院）；Amherst College（阿默斯特学院）；Hampshire College（汉普郡学院）
Over-parameterization is one of the inherent characteristics of modern deep neural networks, which can often be overcome by leveraging regularization methods, such as Dropout. Usually, these methods are applied globally and all the input cases are treated equally. However, given the natural variation of the input space for real-world tasks such as image recognition and natural language understanding, it is unlikely that a fixed regularization pattern will have the same effectiveness for all the input cases. In this work, we demonstrate a method in which the selection of neurons in deep neural …
University of Electronic Science and Technology of China（电子科技大学）；The University of Queensland（昆士兰大学）
Recurrent neural networks (RNNs) are widely used as a memory model for sequence-related problems. Many variants of RNN have been proposed to solve the gradient problems of training RNNs and process long sequences. Although some classical models have been proposed, capturing long-term dependence while responding to short-term changes remains a challenge. To this problem, we propose a new model named Dual Recurrent Neural Networks (DuRNN). The DuRNN consists of two parts to learn the short-term dependence and progressively learn the long-term dependence. The first part is a recurrent neural netw…
