Continual Learning (CL) involves adapting the prior Deep Neural Network (DNN) knowledge to new tasks, without forgetting the old ones. However, modern CL techniques focus on provisioning memory capabilities to existing DNN models rather than designing new ones that are able to adapt according to the task at hand. This paper presents the novel Feedback Continual Learning Vision Transformer (FCL-ViT) that uses a feedback mechanism to generate real-time dynamic attention features tailored to the current task. The FCL-ViT operates in two Phases. In phase 1, the generic image features are produced and determine where the Transformer should attend on the current image. In phase 2, task-specific image features are generated that leverage dynamic attention. To this end, Tunable self-Attention Blocks (TABs) and Task Specific Blocks (TSBs) are introduced that operate in both phases and are responsible for tuning the TABs attention, respectively. The FCL-ViT surpasses state-of-the-art performance on Continual Learning compared to benchmark methods, while retaining a small number of trainable DNN parameters.
Federated Continual Learning via Knowledge Fusion: A Survey
Nanyang Technological University(南洋理工大学);JD iCity, JD Technology(京东科技京东智城);JD Intelligent Cities Research & Institute of Artificial Intelligence, Southwest Jiaotong University(西南交通大学京东智慧城市研究院与人工智能研究所);School of Computing and Artificial Intelligence, Southwest Jiaotong University(西南交通大学计算机与人工智能学院)
Federated Continual Learning: Concepts, Challenges, and Solutions
Faculty of Computer Science, University of New Brunswick(新 Brunswick大学计算机科学学院);Department of Electrical and Computer Engineering, University of Windsor(温莎大学电气与计算机工程系)
Accurate Forgetting for Heterogeneous Federated Continual Learning
Institute for Artificial Intelligence, Tsinghua University(清华大学人工智能研究院);Beijing National Research Center for Information Science and Technology(北京信息科学与技术国家研究中心);The University of Auckland(奥克兰大学);RIKEN(理化学研究所);Hong Kong Baptist University(香港浸会大学);DataCanvas Technology Co., Ltd.(数帆科技有限公司);The University of Tokyo(东京大学)
Handling Spatial-Temporal Data Heterogeneity for Federated Continual Learning via Tail Anchor
Southwestern University of Finance and Economics(西南财经大学);University of Electronic Science and Technology of China(电子科技大学);WeBank(微众银行);Southwest Jiaotong University(西南交通大学);Hong Kong University of Science and Technology(香港科技大学)
Energy-Efficient and Real-Time Sensing for Federated Continual Learning via Sample-Driven Control
School of Electrical and Electronic Engineering, Hanoi University of Science and Technology(电子与电气工程学院,河内科学技术大学);Department of Intelligent Computing and Data Science, VinUniversity(智能计算与数据科学系,文大学);Department of Electrical and Computer Engineering, University of Saskatchewan(电气与计算机工程系,萨斯喀彻温大学);School of Electrical and Data Engineering, University of Technology Sydney(电气与数据工程学院,悉尼技术大学);School of Electricals and Electronics, Hanoi University of Science and Technology(电子与电气工程学院,河内科学技术大学);School of Computer Science and Statistics, Trinity College Dublin(计算机科学与统计学学院,都柏林信任学院)
Variational Bayes for Federated Continual Learning
National Engineering Research Center for Big Data Technology and System(国家大数据技术与系统工程研究中心);School of Computer Science and Technology, Huazhong University of Science and Technology(华中科技大学计算机科学与技术学院);Lehigh University(里海大学);Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学);Tencent AI Lab(腾讯AI Lab)
From Selection to Scheduling: Federated Geometry-Aware Correction Makes Exemplar Replay Work Better under Continual Dynamic Heterogeneity
School of Software, Shandong University(山东大学软件学院);College of Computing and Data Science, Nanyang Technological University(南洋理工大学计算与数据科学学院);School of Computer Science and Technology, Harbin Institute of Technology(哈尔滨工业大学计算机科学与技术学院)
Munsit at NADI 2025 Shared Task 2: Pushing the Boundaries of Multidialectal Arabic ASR with Weakly Supervised Pretraining and Continual Supervised Fine-tuning
Visual Prompt Tuning in Null Space for Continual Learning
School of Computer Science, Northwestern Polytechnical University(西北工业大学计算机学院);School of Telecommunications Engineering, Xidian University(西安电子科技大学通信工程学院)
针对持续学习中的灾难性遗忘,本文提出在 ViT 提示梯度的零空间方向进行正交投影更新,并给出理论条件与近似实现,在多个类别增量基准上优于现有方法。