MagMax: Leveraging Model Merging for Seamless Continual Learning
IDEAS NCBR；Warsaw University of Technology（华沙工业大学）；Autonomous University of Barcelona（巴塞罗那自治大学）；Computer Vision Center（计算机视觉中心）；Tooploox；Gdańsk University of Technology（格但斯克工业大学）
Daniel Marczak, Bartłomiej Twardowski, Tomasz Trzciński, Sebastian Cygert
This paper introduces a continual learning approach named MagMax, which utilizes model merging to enable large pre-trained models to continuously learn from new data without forgetting previously acquired knowledge. Distinct from traditional continual learning methods that aim to reduce forgetting during task training, MagMax combines sequential fine-tuning with a maximum magnitude weight selection for effective knowledge integration across tasks. Our initial contribution is an extensive examination of model merging techniques, revealing that simple approaches like weight averaging and random weight selection surprisingly hold up well in various continual learning contexts. More importantly, we present MagMax, a novel model-merging strategy that enables continual learning of large pre-trained models for successive tasks. Our thorough evaluation demonstrates the superiority of MagMax in various scenarios, including class- and domain-incremental learning settings. The code is available at this URL: https://github.com/danielm1405/magmax.
IDER: IDempotent Experience Replay for Reliable Continual Learning
School of Computer Science, Shanghai Jiao Tong University（上海交通大学计算机科学学院）；Shanghai Innovation Institute（上海创新研究院）；Lehigh University（莱斯大学）；State Key Laboratory of General Artificial Intelligence, BIGAI（通用人工智能国家重点实验室，BIGAI）；MIFA Lab（MIFA实验室）
Adapt Your Teacher: Improving Knowledge Distillation for Exemplar-free Continual Learning
Warsaw University of Technology（华沙理工大学）；IDEAS NCBR；Jagiellonian University（雅盖隆大学）；Gdańsk University of Technology（格但斯克理工大学）；Autonomous University of Barcelona（巴塞罗那自治大学）；Computer Vision Center（计算机视觉中心）；Tooploox
University of Bremen（不来梅大学）；Jeonbuk National University（全北国立大学）；TU Darmstadt（达姆施塔特工业大学）；RIKEN Center for Advanced Intelligence Project（理化学研究所先进智能项目中心）；Hessian Center for AI (hessian.AI)（黑森人工智能中心）；University College London（伦敦大学学院）
On Generalizing Beyond Domains in Cross-Domain Continual Learning
The Australian National University（澳大利亚国立大学）；NEC Labs America（美国NEC实验室）；Monash University（莫纳什大学）；Data61（Data61研究院）；Phiar Technologies（Phiar科技公司）；University of California, San Diego（加州大学圣地亚哥分校）
LLMs Can Evolve Continually on Modality for X-Modal Reasoning
Dalian University of Technology（大连理工大学）；Huawei Noah’s Ark Lab（华为诺亚方舟实验室）；Tsinghua University（清华大学）；The Hong Kong University of Science and Technology（香港科技大学）
FastODT: A tree-based framework for efficient continual learning
Masaryk University（马萨里克大学）；Czech Republic（捷克共和国）；Warsaw University of Technology（华沙技术大学）；Poland（波兰）；Mycroft Mind；New York University（纽约大学）；New York, NY, USA（纽约市，纽约州，美国）；Analog AI；Abu Dhabi, UAE（阿布扎赫，阿联酋）
Ferret: An Efficient Online Continual Learning Framework under Varying Memory Constraints
Sichuan University（四川大学）；National University of Singapore（新加坡国立大学）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Huazhong University of Science and Technology（华中科技大学）
Diffusion-Driven Data Replay: A Novel Approach to Combat Forgetting in Federated Class Continual Learning
China University of Petroleum（中国石油大学）；WeBank（微众银行）；Pazhou Laboratory（琶洲实验室）；South China University of Technology（华南理工大学）；The Hong Kong University of Science and Technology（香港科技大学）
Towards General Purpose Medical AI: Continual Learning Medical Foundation Model
Beijing University of Posts and Telecommunications（北京邮电大学）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；West China Hospital, Sichuan University（四川大学华西医院）；Shanghai Jiao Tong University（上海交通大学）；Sichuan University Pittsburgh Institute（四川大学匹兹堡学院）
Unlocking Continual Learning Abilities in Language Models
School of Computing and Data Science, The University of Hong Kong（香港大学计算与数据科学学院）；CUHK-SZ（香港中文大学（深圳））；Tsinghua University（清华大学）；University of Edinburgh（爱丁堡大学）；NVIDIA（英伟达公司）；HKUST（香港科技大学）；ICT, Chinese Academy of Sciences（中国科学院信息技术研究所）
Learning from the Past: Continual Meta-Learning via Bayesian Graph Modeling
The University of Queensland（昆士兰大学）；Harbin Institute of Technology（哈尔滨工业大学）；Pengcheng Laboratory（鹏城实验室）；University of Electronic Science and Technology of China（电子科技大学）
Unveiling the Tapestry: the Interplay of Generalization and Forgetting in Continual Learning
Nanyang Technological University（南洋理工大学）；Agency for Science, Technology and Research（科学、技术与研究局）；Hefei University of Technology（合肥工业大学）；Sichuan University（四川大学）
Generative Model-Aided Continual Learning for CSI Feedback in FDD mMIMO-OFDM Systems
College of Information Science and Technology, Donghua University（东华大学信息科学与技术学院）；Department of Information and Computer Science, Keio University（Keio大学信息与计算机科学系）；School of Computing and Information Technology, Great Bay University（Great Bay大学计算与信息科技学院）；Great Bay Institute for Advanced Study (GBIAS)（Great Bay先进研究 institutes）；Department of Engineering, Nottingham Trent University（Nottingham Trent大学工程系）
Learning from Mistakes: Loss-Aware Memory Enhanced Continual Learning for LiDAR Place Recognition
Shanghai Research Institute for Intelligent Autonomous System, Tongji University（上海智能自主系统研究院，同济大学）；Department of Computer Science and Technology, School of Electronics and Information Engineering, Tongji University（计算机科学与技术系，电子信息工程学院，同济大学）；MOE Key Lab of Embedded System and Service Computing, Tongji University（教育部嵌入式系统与服务计算重点实验室，同济大学）；Institute of Intelligent Vehicles, Tongji University（智能车辆研究院，同济大学）；College of Surveying and Geo-Informatics, Tongji University（测绘地理信息学院，同济大学）
Dynamic Dual Buffer with Divide-and-Conquer Strategy for Online Continual Learning
Department of Computing, Imperial College London（帝国理工学院计算机系）；Department of Bioengineering, Imperial College London（帝国理工学院生物工程系）；School of Information and Software Engineering, University of Electronic Science and Technology of China（电子科技大学信息与软件工程学院）
Continual Learning for Segment Anything Model Adaptation
Southern University of Science and Technology（南方科技大学）；City University of Hong Kong（香港城市大学）；Hong Kong University of Science and Technology（香港科技大学）；Tencent YouTu Lab（腾讯优图实验室）
针对动态流式数据场景，提出持续 SAM 适应基准 CoSAM 和域适配器混合算法 MoDA，通过全局特征与辅助令牌提取分离特征，实现持续分割并保持知识，优于现有方法。
Efficient Continual Learning in Keyword Spotting using Binary Neural Networks
Department of Computer and Electrical Engineering, Mid Sweden University（计算机与电气工程系，Mid Sweden大学）；Department of Information Technology and Electrical Engineering, ETH Zürich（信息科技与电气工程系，ETH Zürich）
Enhancing Visual Continual Learning with Language-Guided Supervision
State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；Centre for Artificial Intelligence and Robotics, HK Institute of Science & Innovation, Chinese Academy of Sciences（中国科学院香港创新研究院人工智能与机器人中心）
Regression-aware Continual Learning for Android Malware Detection
Department of Electrical and Electronic Engineering, University of Cagliari, Italy（电子与电气工程系，卡利阿里大学）；Department of Computer, Control and Management Engineering, Sapienza University, Rome, Italy（计算机、控制与管理工程系，萨皮恩扎大学）；Department of Informatics, Bioengineering, Robotics, and Systems Engineering, University of Genoa, Italy（信息学、生物工程、机器人学和系统工程系，热那亚大学）；CINI, Italy（CINI，意大利）
