Despite significant advances in graph representation learning, little attention has been paid to the more practical continual learning scenario in which new categories of nodes (e.g., new research areas in citation networks, or new types of products in co-purchasing networks) and their associated edges are continuously emerging, causing catastrophic forgetting on previous categories. Existing methods either ignore the rich topological information or sacrifice plasticity for stability. To this end, we present Hierarchical Prototype Networks (HPNs) which extract different levels of abstract knowledge in the form of prototypes to represent the continuously expanded graphs. Specifically, we first leverage a set of Atomic Feature Extractors (AFEs) to encode both the elemental attribute information and the topological structure of the target node. Next, we develop HPNs to adaptively select relevant AFEs and represent each node with three levels of prototypes. In this way, whenever a new category of nodes is given, only the relevant AFEs and prototypes at each level will be activated and refined, while others remain uninterrupted to maintain the performance over existing nodes. Theoretically, we first demonstrate that the memory consumption of HPNs is bounded regardless of how many tasks are encountered. Then, we prove that under mild constraints, learning new tasks will not alter the prototypes matched to previous data, thereby eliminating the forgetting problem. The theoretical results are supported by experiments on five datasets, showing that HPNs not only outperform state-of-the-art baseline techniques but also consume relatively less memory.
Instance-Prototype Affinity Learning for Non-Exemplar Continual Graph Learning
Jiangsu Provincial Joint International Research Laboratory of Medical Information Processing, School of Computer Science and Engineering, Southeast University(江苏省级国际医学信息处理联合研究实验室,计算机科学与工程学院,东南大学);Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications (Southeast University), Ministry of Education, China(新一代人工智能技术及其跨学科应用重点实验室(东南大学),教育部,中国)
Contrastive Continual Learning with Importance Sampling and Prototype-Instance Relation Distillation
Sun Yat-sen University(中山大学);Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学);The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));The Hong Kong University of Science and Technology(香港科技大学)
Prototype-Guided Non-Exemplar Continual Learning for Cross-subject EEG Decoding
Dept. of Artificial Intelligence Korea University Seoul, Republic of Korea(人工智能系 韩国大学首尔共和国);Cognitive Engineering Korea University Seoul, Republic of Korea(认知工程 韩国大学首尔共和国)
Online Continual Learning with Dynamic Label Hierarchies
College of Computer Science(计算机科学学院);Technology, Nanjing University of Aeronautics(技术学院,南京航空航天大学);MIIT Key Laboratory of Pattern Analysis(模式分析 MIIT 实验室);Computer Vision Center, Spain(西班牙计算机视觉中心);Computer Sciences Department, Universitat Autonoma de Barcelona, Spain(巴塞罗那自治大学计算机科学系,西班牙);State Key Laboratory for Novel Software Technology, Nanjing University(新型软件技术国家重点实验室,南京大学);IDEAS Research Institute, Warsaw, Poland(华沙 IDEAS 研究院,波兰);Joint Laboratory of Spatial intelligent Perception(空间智能感知联合实验室)
Leveraging Hierarchical Taxonomies in Prompt-based Continual Learning
VinAI Research(VinAI研究院);New York University(纽约大学);Monash University(莫纳什大学);Hanoi University of Science and Technology(河内科技大学);University of Oregon(俄勒冈大学);The University of Texas at Austin(德克萨斯大学奥斯汀分校)
FPPL: An Efficient and Non-IID Robust Federated Continual Learning Framework
School of Computer Science and Technology, East China Normal University(华东师范大学计算机科学与技术学院);School of Software Engineering, TongJi University(同济大学软件工程学院)
Dynamic Dual Buffer with Divide-and-Conquer Strategy for Online Continual Learning
Department of Computing, Imperial College London(帝国理工学院计算机系);Department of Bioengineering, Imperial College London(帝国理工学院生物工程系);School of Information and Software Engineering, University of Electronic Science and Technology of China(电子科技大学信息与软件工程学院)
The Chinese University of Hong Kong(香港中文大学);National University of Singapore(新加坡国立大学);Yale University(耶鲁大学);The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州))
Brain-Inspired Hierarchical Modularity for General Continual Learning
Tsinghua University(清华大学);IDG/McGovern Institute for Brain Research, Tsinghua University(清华大学IDG/麦戈文脑科学研究院);Tsinghua-Peking Center for Life Sciences(清华-北大生命科学联合中心)
SPECI: Skill Prompts based Hierarchical Continual Imitation Learning for Robot Manipulation
State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences(多模态人工智能系统国家重点实验室,自动化研究所,中国科学院);School of Artificial Intelligence, University of Chinese Academy of Sciences(人工智能学院,中国科学院大学)
Continual Learning for non-stationary regression via Memory-Efficient Replay
atlanTTic - ICLAB - Universidade de Vigo, Escola de Enxeñaría de Telecomunicación, Campus Universitario(atlanTTic - ICLAB - 维戈大学,电信工程学院,大学校区);Centro Tecnolóxico de Telecomunicacións de Galicia (GRADIANT)(加利西亚电信技术中心(GRADIANT));Facultade de Informática, Universidade de Santiago de Compostela(圣地亚哥-德孔波斯特拉大学计算机学院)
Hierarchical Variational Memory for Few-shot Learning Across Domains
University of Amsterdam(阿姆斯特丹大学);Inception Institute of Artificial Intelligence(Inception人工智能研究院);National Center for Artificial Intelligence, Saudi Data and Artificial Intelligence Authority(沙特数据与人工智能局国家人工智能中心)
Rethinking Continual Learning with Progressive Neural Collapse
Department of Computer Science, University of Houston, USA(德克萨斯大学休斯顿分校计算机科学系);Department of Computer Science, University of North Carolina at Charlotte, USA(北卡罗来纳大学夏洛特分校计算机科学系)
Continual Adapter Tuning with Semantic Shift Compensation for Class-Incremental Learning
HUST AI & Visual Learning Lab (HAIV Lab)(华中科技大学人工智能与视觉学习实验室);Huazhong University of Science and Technology (HUST)(华中科技大学);Department of Computer Science(计算机科学系);Boston University(波士顿大学);ByteDance, Inc.(字节跳动公司)
HiDe-LLaVA: Hierarchical Decoupling for Continual Instruction Tuning of Multimodal Large Language Model
School of Advanced Interdisciplinary Sciences, UCAS(UCAS先进交叉科学学院);MAIS, CASIA(CASIA人工智能研究所);School of Artificial Intelligence, UCAS(UCAS人工智能学院);Centre for Artificial Intelligence and Robotics, HKISI-CAS(HKISI-CAS人工智能与机器人中心);FKLPRIU, School of Computer and Information Engineering, Xiamen University of Technology(厦门理工学院计算机与信息工程学院)