Lifelong machine learning or continual learning models attempt to learn incrementally by accumulating knowledge across a sequence of tasks. Therefore, these models learn better and faster. They are used in various intelligent systems that have to interact with humans or any dynamic environment e.g., chatbots and self-driving cars. Memory-less approach is more often used with deep neural networks that accommodates incoming information from tasks within its architecture. It allows them to perform well on all the seen tasks. These models suffer from semantic drift or the plasticity-stability dilemma. The existing models use Minkowski distance measures to decide which nodes to freeze, update or duplicate. These distance metrics do not provide better separation of nodes as they are susceptible to high dimensional sparse vectors. In our proposed approach, we use angular distance to evaluate the semantic drift in individual nodes that provide better separation of nodes and thus better balancing between stability and plasticity. The proposed approach outperforms state-of-the art models by maintaining higher accuracy on standard datasets.
Overcoming Domain Drift in Online Continual Learning
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);Suzhou University of Science and Technology(苏州科技大学);Youtu Lab, Tencent(腾讯优图实验室);College of Intelligence and Computing, Tianjin University(天津大学智能与计算学部)
Keep Rehearsing and Refining: Lifelong Learning Vehicle Routing under Continually Drifting Tasks
Center of Data Science and Artificial Intelligence & School of Engineering and Computer Science(数据科学与人工智能中心及工程与计算机科学学院);Victoria University of Wellington(怀特维尔大学);School of Data Science(数据科学学院)
本文提出了一种应对持续漂移任务的终身学习方法,通过Dual Replay with Experience Enhancement框架提升学习效率并缓解灾难性遗忘。
CoMemNet: A Continual Memory Network with Drift-Aware Sampling for Traffic Prediction
College of Computer Science, Shanghai Jiao Tong University(上海交通大学计算机科学学院);College of Computer Science and Technology, Zhejiang University of Technology(浙江工业大学计算机科学与技术学院);School of Automation, Nanjing University of Science and Technology(南京理工大学自动化学院)
Outlook Towards Deployable Continual Learning for Particle Accelerators
Thomas Jefferson National Accelerator Facility(托马斯杰斐逊国家加速器设施);University of Houston(休斯顿大学);SLAC National Laboratory(SLAC国家实验室);Oak Ridge National Laboratory(橡树岭国家实验室)
Drift and Dependence: Layer-wise Information-Theoretic Bounds for Replay-Based Continual Learning
School of Computer Science and Technology, Xi’an Jiaotong University(西安交通大学计算机科学与技术学院);Department of Computer Science and Technology, Tsinghua University(清华大学计算机科学与技术系)
LoRA Subtraction for Drift-Resistant Space in Exemplar-Free Continual Learning
School of Artificial Intelligence, Sun Yat-sen University(中山大学人工智能学院);Key Laboratory of Intelligent Assessment Technology for Sustainable Tourism, Ministry of Culture and Tourism, Sun Yat-sen University(文化和旅游部可持续旅游智能评估技术重点实验室);Key Laboratory of Machine Intelligence and Advanced Computing, Ministry of Education, China(教育部机器智能与先进计算重点实验室)
PASs-MoE: Mitigating Misaligned Co-drift among Router and Experts via Pathway Activation Subspaces for Continual Learning
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);University of Chinese Academy of Sciences(中国科学院大学);National University of Singapore(新加坡国立大学);Southeast University, Nanjing, China(南京东南大学);Wuhan AI Research, Wuhan, China(武汉人工智能研究院)
Query Drift Compensation: Enabling Compatibility in Continual Learning of Retrieval Embedding Models
Computer Vision Center, Barcelona, Spain(巴塞罗那计算机视觉中心);Department of Computer Science, Universitat Autònoma de Barcelona, Spain(巴塞罗那自治大学计算机科学系);IDEAS Research Center, Warsaw, Poland(华沙IDEAS研究中心)
The Chinese University of Hong Kong(香港中文大学);National University of Singapore(新加坡国立大学);Yale University(耶鲁大学);The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州))
Dual-LoRA and Quality-Enhanced Pseudo Replay for Multimodal Continual Food Learning
College of Computer Science and Artificial Intelligence, Fudan University, China(复旦大学计算机科学与人工智能学院);Singapore Management University, Singapore(新加坡国立管理学院)
ICICLE: Interpretable Class Incremental Continual Learning
Jagiellonian University(雅盖隆大学);Ardigen SA(阿迪根公司);Autonomous University of Barcelona(巴塞罗那自治大学);Computer Vision Center(计算机视觉中心);IDEAS NCBR(IDEAS国家计算智能研究中心)
Exploring the Impact of Parameter Update Magnitude on Forgetting and Generalization of Continual Learning
Institute of Intelligent Information Processing, Shanxi University(山西大学智能信息处理研究所);Alliance Manchester Business School, The University of Manchester(曼彻斯特大学阿利安斯商学院)