TinyCL: An Efficient Hardware Architecture for Continual Learning on Autonomous Systems
Department of Electronics and Telecommunications, Politecnico di Torino（电子工程与电信系，都灵理工学院）；eBrain Lab, Division of Engineering, New York University Abu Dhabi（eBrain实验室，工程系，纽约大学阿布扎克分校）
Eugenio Ressa, Alberto Marchisio, Maurizio Martina, Guido Masera, Muhammad Shafique
备注：
Accepted at the 2025 International Joint Conference on Neural Networks (IJCNN), Rome, Italy, July 2025
The Continuous Learning (CL) paradigm consists of continuously evolving the parameters of the Deep Neural Network (DNN) model to progressively learn to perform new tasks without reducing the performance on previous tasks, i.e., avoiding the so-called catastrophic forgetting. However, the DNN parameter update in CL-based autonomous systems is extremely resource-hungry. The existing DNN accelerators cannot be directly employed in CL because they only support the execution of the forward propagation. Only a few prior architectures execute the backpropagation and weight update, but they lack the control and management for CL. Towards this, we design a hardware architecture, TinyCL, to perform CL on resource-constrained autonomous systems. It consists of a processing unit that executes both forward and backward propagation, and a control unit that manages memory-based CL workload. To minimize the memory accesses, the sliding window of the convolutional layer moves in a snake-like fashion. Moreover, the Multiply-and-Accumulate units can be reconfigured at runtime to execute different operations. As per our knowledge, our proposed TinyCL represents the first hardware accelerator that executes CL on autonomous systems. We synthesize the complete TinyCL architecture in a 65 nm CMOS technology node with the conventional ASIC design flow. It executes 1 epoch of training on a Conv + ReLU + Dense model on the CIFAR10 dataset in 1.76 s, while 1 training epoch of the same model using an Nvidia Tesla P100 GPU takes 103 s, thus achieving a 58x speedup, consuming 86 mW in a 4.74 mm2 die.
CLANE: Continual Learning of Actions on Neuromorphic Hardware from Event Cameras
Institute of Informatics, University of Munich (LMU)（慕尼黑大学信息学院）；fortiss GmbH, Neuromorphic Computing（fortiss GmbH 神经形态计算部门）；Technical University of Munich, TUM School of CIT（慕尼黑技术大学 CIT 学院）；Intel Labs, Intel Corporation（英特尔实验室，英特尔公司）；Institute of Computational Life Sciences (ICLS), Zurich University of Applied Sciences (ZHAW)（应用科学大学（ZHAW）计算生命科学研究所）；Technische Universität Ilmenau, Resource-Efficient Artificial Intelligence Group（伊门豪大学资源高效人工智能小组）；Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心）；German Research Centre for Artificial Intelligence (DFKI)（德国人工智能研究中心）
Critical Patch-Aware Sparse Prompting with Decoupled Training for Continual Learning on the Edge
School of Computer Science and Engineering, Chung-Ang University（中央大学计算机科学与工程学院）；Department of Artificial Intelligence, Chung-Ang University（中央大学人工智能系）
Continual Learning with Neuromorphic Computing: Foundations, Methods, and Emerging Applications
Electrical and Communication Engineering Department, United Arab Emirates University (UAEU), Al Ain, United Arab Emirates（阿联酋大学电子与通信工程系）；eBRAIN Lab, New York University (NYU) Abu Dhabi, United Arab Emirates（纽约大学阿布扎赫实验室）；National University of Sciences and Technology (NUST), Pakistan（巴基斯坦国立科学与技术大学）
EDAPT: Towards Calibration-Free BCIs with Continual Online Adaptation
Excellence Cluster Machine Learning, University of Tübingen（卓越集群机器学习中心，图宾根大学）；Tübingen AI Center, University of Tübingen（图宾根人工智能中心，图宾根大学）；Department of Neurology and Stroke, University of Tübingen（神经学与中风部门，图宾根大学）；Hertie Institute for Clinical Brain Research, University of Tübingen（赫尔特研究所，图宾根大学）；Department of Empirical Inference, Max Planck Institute for Intelligent Systems（经验推理部门，智能系统马克斯·普朗克研究所）；Hertie Institute for AI in Brain Health, Tübingen（赫尔特人工智能与脑健康研究所，图宾根）
CVPR 2020 Continual Learning in Computer Vision Competition: Approaches, Results, Current Challenges and Future Directions
University of Bologna（博洛尼亚大学）；ContinualAI Research（持续人工智能研究所）；Element AI（元素人工智能公司）；MILA（蒙特利尔学习算法研究所）；ByteDance AI Labs（字节跳动人工智能实验室）；University of Bristol（布里斯托大学）；Tokyo Institute of Technology（东京工业大学）；AIST RWBC-OIL（日本产业技术综合研究所实世界大数据与计算开放创新实验室）；Chinese Academy of Sciences（中国科学院）；University of Toronto（多伦多大学）；University of Hamburg（汉堡大学）；University of Cambridge（剑桥大学）；Google AI（谷歌人工智能）
CLoRA: Parameter-Efficient Continual Learning with Low-Rank Adaptation
Augmented Vision Group, German Research Center for Artificial Intelligence (DFKI)（德国人工智能研究中心增强视觉组）；RPTU – University of Kaiserslautern-Landau（凯撒斯劳滕-兰道大学）
MetaCLBench: Meta Continual Learning Benchmark on Resource-Constrained Edge Devices
Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；Samsung AI Center-Cambridge（三星AI中心-剑桥）；The Hong Kong Polytechnic University（香港理工大学）
Neuromorphic Continual Learning for Sequential Deployment of Nuclear Plant Monitoring Systems
University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；University of Texas at El Paso（德克萨斯大学埃尔帕索分校）；National Center for Supercomputing Applications（国家超级计算应用中心）
Online Continual Learning on Intel Loihi 2 via a Co-designed Spiking Neural Network
Institute of Informatics, University of Munich (LMU)（慕尼黑大学信息学院）；Intel Labs, Intel Corporation（英特尔实验室）；Southwest Research Institute（西南研究 institute）；Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心）；German Research Centre for Artificial Intelligence (DFKI)（德国人工智能研究中心）；Institute of Computational Life Sciences (ICLS)（计算生命科学研究所）；Zurich University of Applied Sciences (ZHAW)（苏黎世应用科学大学）
Gradient-Free Continual Learning in Spiking Neural Networks via Inter-Spike Interval Regularization
University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Indian Institute of Technology Delhi（印度理工学院德里）；University of Texas at El Paso（得克萨斯大学埃尔帕索分校）；National Center for Supercomputing Applications（国家超级计算应用中心）
Temporal Memory for Resource-Constrained Agents: Continual Learning via Stochastic Compress-Add-Smooth
Graduate Inter-Disciplinary Program in Applied Mathematics, & Department of Mathematics, University Arizona, Tucson, AZ 85721（亚利桑那大学应用数学跨学科研究生项目与数学系，图森，AZ 85721）
Self-Organising Memristive Networks as Physical Learning Systems
Theoretical Division (Condensed Matter and Complex Systems, T-4)（洛斯阿拉莫斯国家实验室理论部）；Advanced Materials Metrology and Life Sciences Division, INRiM (Istituto Nazionale di Ricerca Metrologica)（先进材料计量与生命科学部，INRiM（意大利国家计量研究院））；University of California, Los Angeles, California NanoSystems Institute（加州大学洛杉矶分校，加州纳米系统研究所）；Department of Applied Science and Technology, Politecnico di Torino（托里尼 Politecnico 工程技术系）；The MacDiarmid Institute for Advanced Materials and Nanotechnology, School of Physical and Chemical Sciences, Te Kura Mate, University of Canterbury（麦克迪尔马特高级材料与纳米技术研究所，坎特伯雷大学物理与化学科学学院，Te Kura Mate）；School of Physics, Sydney Nano Institute and Centre for Complex Systems, University of Sydney（悉尼大学物理学院，悉尼纳米研究所和复杂系统中心）
Neuromorphic Cybersecurity with Semi-supervised Lifelong Learning
School of EECS（电子工程与科学学院）；The Pennsylvania State University（宾夕法尼亚州立大学）；Dept. of EECS（电子工程与科学系）；University of Michigan（密歇根大学）；Quantum Ventura, Inc.（Quantum Ventura 公司）；University of Texas at Arlington Research Institute（德克萨斯大学阿灵顿研究 institute）
