Merging Models on the Fly Without Retraining: A Sequential Approach to Scalable Continual Model Merging
Wuhan University（武汉大学）；Northeastern University（东北大学）；Sun Yat-sen University（中山大学）；Beijing Institute of Technology（北京理工大学）；Nanyang Technological University（南洋理工大学）
Anke Tang, Enneng Yang, Li Shen, Yong Luo, Han Hu, Bo Du, Dacheng Tao
Deep model merging represents an emerging research direction that combines multiple fine-tuned models to harness their specialized capabilities across different tasks and domains. Current model merging techniques focus on merging all available models simultaneously, with weight interpolation-based methods being the predominant approaches. However, these conventional approaches are not well-suited for scenarios where models become available sequentially, and they often suffer from high memory requirements and potential interference between tasks. In this study, we propose a training-free projection-based continual merging method that processes models sequentially through orthogonal projections of weight matrices and adaptive scaling mechanisms. Our method operates by projecting new parameter updates onto subspaces orthogonal to existing merged parameter updates while using an adaptive scaling mechanism to maintain stable parameter distances, enabling efficient sequential integration of task-specific knowledge. Our approach maintains constant memory complexity to the number of models, minimizes interference between tasks through orthogonal projections, and retains the performance of previously merged models through adaptive task vector scaling. Extensive experiments on CLIP-ViT models demonstrate that our method achieves a 5-8% average accuracy improvement while maintaining robust performance in different task orderings.
Fly-CL: A Fly-Inspired Framework for Enhancing Efficient Decorrelation and Reduced Training Time in Pre-trained Model-based Continual Representation Learning
Department of Automation, Tsinghua University（清华大学自动化系）；Academy of Medical Engineering and Translational Medicine, Tianjin University（天津大学医学工程与转化医学学院）
FlyPrompt: Brain-Inspired Random-Expanded Routing with Temporal-Ensemble Experts for General Continual Learning
School of Life Sciences, IDG/McGovern Institute for Brain Research, Tsinghua University（生命科学学院，IDG/麦克格雷文脑科学研究院，清华大学）；Department of Psychological and Cognitive Sciences, Tsinghua University（心理学与认知科学系，清华大学）；School of Medicine, Shenzhen Campus of Sun Yat-Sen University（中山大学深圳校区医学学院）
Multi-Level Safety Continual Projection for Fine-Tuned Large Language Models without Retraining
Beijing Key Laboratory of Safe AI and Superalignment（北京安全人工智能与超对齐重点实验室）；Beijing Institute of AI Safety and Governance（北京人工智能安全与治理研究院）；Brain-inspired Cognitive AI Lab, Institute of Automation, Chinese Academy of Sciences（脑启发认知人工智能实验室，中国科学院自动化研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Long-term AI（长期人工智能）
Suzhou University of Science and Technology（苏州科技大学）；Tianjin University（天津大学）
Rehearsal, retraining on a stored small data subset of old tasks, has been proven effective in solving catastrophic forgetting in continual learning. However, due to the sampled data may have a large bias towards the original dataset, retraining them is susceptible to driving continual domain drift of old tasks in feature space, resulting in forgetting. In this paper, we focus on tackling the continual domain drift problem with centroid distance distillation. First, we propose a centroid caching mechanism for sampling data points based on constructed centroids to reduce the sample bias in rehe…
SAE-FD: Sparse Autoencoder Feature Distillation for Continual Learning of Large Language Models
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；Nanjing University of Aeronautics and Astronautics（南京航空航天大学）；The 63rd Research Institute, National University of Defense Technology, Nanjing（国防科技大学第六十三研究所，南京）
FreeMOCA: Memory-Free Continual Learning for Malicious Code Analysis
Department of Computer Engineering（计算机工程系）；Amirkabir University of Technology（阿姆irkabir技术大学）；Division of Artificial Intelligence & Software（人工智能与软件系）；Ewha Womans University（成均馆大学）；Department of Computer Science（计算机科学系）；University of Texas at El Paso（德克萨斯理工大学）
Continual Learning, also known as Lifelong Learning, aims to continually learn from new data as it becomes available. While prior research on continual learning in automatic speech recognition has focused on the adaptation of models across multiple different speech recognition tasks, in this paper we propose an experimental setting for \textit{online continual learning} for automatic speech recognition of a single task. Specifically focusing on the case where additional training data for the same task becomes available incrementally over time, we demonstrate the effectiveness of performing inc…
Continual Learning for non-stationary regression via Memory-Efficient Replay
atlanTTic - ICLAB - Universidade de Vigo, Escola de Enxeñaría de Telecomunicación, Campus Universitario（atlanTTic - ICLAB - 维戈大学，电信工程学院，大学校区）；Centro Tecnolóxico de Telecomunicacións de Galicia (GRADIANT)（加利西亚电信技术中心（GRADIANT））；Facultade de Informática, Universidade de Santiago de Compostela（圣地亚哥-德孔波斯特拉大学计算机学院）
Self-Supervised Continual Graph Learning in Adaptive Riemannian Spaces
North China Electric Power University（华北电力大学）；Beijing University of Posts and Telecommunications（北京邮电大学）；Beihang University（北京航空航天大学）；University of Illinois at Chicago（伊利诺伊大学芝加哥分校）
University of Southern California（南加州大学）；Amazon（亚马逊）
In order to be effective general purpose machines in real world environments, robots not only will need to adapt their existing manipulation skills to new circumstances, they will need to acquire entirely new skills on-the-fly. A great promise of continual learning is to endow robots with this ability, by using their accumulated knowledge and experience from prior skills. We take a fresh look at this problem, by considering a setting in which the robot is limited to storing that knowledge and experience only in the form of learned skill policies. We show that storing skill policies, careful pr…
Seoul National University（首尔大学）；Massachusetts Institute of Technology（麻省理工学院）
Incremental class learning, a scenario in continual learning context where classes and their training data are sequentially and disjointedly observed, challenges a problem widely known as catastrophic forgetting. In this work, we propose a novel incremental class learning method that can significantly reduce memory overhead compared to previous approaches. Apart from conventional classification scheme using softmax, our model bases on an autoencoder to extract prototypes for given inputs so that no change in its output unit is required. It stores only the mean of prototypes per class to perfor…
In many real-world scenarios, data to train machine learning models become available over time. However, neural network models struggle to continually learn new concepts without forgetting what has been learnt in the past. This phenomenon is known as catastrophic forgetting and it is often difficult to prevent due to practical constraints, such as the amount of data that can be stored or the limited computation sources that can be used. Moreover, training large neural networks, such as Transformers, from scratch is very costly and requires a vast amount of training data, which might not be ava…
Replay in neural networks involves training on sequential data with memorized samples, which counteracts forgetting of previous behavior caused by non-stationarity. We present a method where these auxiliary samples are generated on the fly, given only the model that is being trained for the assessed objective, without extraneous buffers or generator networks. Instead the implicit memory of learned samples within the assessed model itself is exploited. Furthermore, whereas existing work focuses on reinforcing the full seen data distribution, we show that optimizing for not forgetting calls for …
Multi-Task Learning (MTL) is widely-accepted in Natural Language Processing as a standard technique for learning multiple related tasks in one model. Training an MTL model requires having the training data for all tasks available at the same time. As systems usually evolve over time, (e.g., to support new functionalities), adding a new task to an existing MTL model usually requires retraining the model from scratch on all the tasks and this can be time-consuming and computationally expensive. Moreover, in some scenarios, the data used to train the original training may be no longer available, …
