An intelligent system capable of continual learning is one that can process and extract knowledge from potentially infinitely long streams of pattern vectors. The major challenge that makes crafting such a system difficult is known as catastrophic forgetting - an agent, such as one based on artificial neural networks (ANNs), struggles to retain previously acquired knowledge when learning from new samples. Furthermore, ensuring that knowledge is preserved for previous tasks becomes more challenging when input is not supplemented with task boundary information. Although forgetting in the context of ANNs has been studied extensively, there still exists far less work investigating it in terms of unsupervised architectures such as the venerable self-organizing map (SOM), a neural model often used in clustering and dimensionality reduction. While the internal mechanisms of SOMs could, in principle, yield sparse representations that improve memory retention, we observe that, when a fixed-size SOM processes continuous data streams, it experiences concept drift. In light of this, we propose a generalization of the SOM, the continual SOM (CSOM), which is capable of online unsupervised learning under a low memory budget. Our results, on benchmarks including MNIST, Kuzushiji-MNIST, and Fashion-MNIST, show almost a two times increase in accuracy, and CIFAR-10 demonstrates a state-of-the-art result when tested on (online) unsupervised class incremental learning setting.
Inria Bordeaux Sud-Ouest（波尔多西南法国国立信息学及自动化研究院）；Institut des Maladies Neurodégénératives, Université de Bordeaux（波尔多大学神经退行性疾病研究所）；Université de Bordeaux（波尔多大学）；LaBRI, Université de Bordeaux（波尔多大学LaBRI实验室）；Institut Polytechnique de Bordeaux（波尔多理工学院）；adNomus Inc.（adNomus公司）
A Feature-Major Codebook for Memory-Efficient Sparse-Binary Self-Organizing Maps: Scaling a MEDLINE Atlas to 1.05 Million Neurons on a Single Consumer GPU
College of Medicine and Dentistry, James Cook University（詹姆斯库克大学医学与牙科学院）
Adaptive SpikeDeep-Classifier: Self-organizing and self-supervised machine learning algorithm for online spike sorting
Ruhr-West University of Applied Science（鲁尔-西应用科学大学）；University hospital Knappschaftskrankenhaus Bochum GmbH（波鸿矿工医院有限公司大学医院）；Ruhr-University Bochum（鲁尔大学波鸿）
Self-orthogonalizing attractor neural networks emerging from the free energy principle
Center for Translational Neuro- and Behavioral Sciences (C-TNBS), University Medicine Essen, Germany（转化神经与行为科学中心（C-TNBS），埃森大学医学中心，德国）；Queen Square Institute of Neurology, University College London, WC1N 3AR, UK（皇后广场神经病学研究所，伦敦大学学院，英国）；VERSES, Los Angeles, CA 90067, USA（VERSES，美国加利福尼亚州洛杉矶90067）
