终身学习者必须认识到随着时间的推移而演变的概念词汇。一个常见但未充分探索的场景是使用不断改进/扩展旧类的类标签进行学习。例如,人类在狗繁殖之前就学会了识别${\tt dog}$。在实际环境中,数据集$\textit{versioning}$经常引入对本体的细化,例如,随着自主操作扩展到新的城市,自主车辆基准将先前的${\tt vehicle}$类细化为${\tt school-bus}$。本文形式化了一个研究$\textit{Learning with Evolving Class Ontology}$(LECO)问题的协议。LECO需要在不同的时间段(TP)中学习分类器;每个TP引入新的“精细”标签本体,其细化旧的“粗略”标签本体(例如,改进先前${\tt dog}$的狗品种)。LECO探索了诸如是否注释新数据或重新标记旧数据、如何利用粗糙标签以及是否微调先前TP的模型或从头开始训练等问题。为了回答这些问题,我们利用相关问题的见解,如类增量学习。我们通过图像分类(CIFAR和iNaturalist)和语义分割(Mapillary)的镜头在LECO协议下验证它们。我们的实验得出了令人惊讶的结论;虽然目前的现状是用新的本体(如COCO到LVIS或Mapillary1.2到2.0)重新标记现有的数据集,但LECO证明了一个更好的策略是用新的本体注释$\textit{new}$数据。然而,这会产生一个新旧标签不一致的聚合数据集,使学习变得复杂。为了应对这一挑战,我们采用了半监督和部分标签学习的方法。这样的策略可以令人惊讶地接近最优,接近一个“预言机”,它在用最新本体详尽标记的聚合数据集上学习。
英文摘要:
Lifelong learners must recognize concept vocabularies that evolve over time. A common yet underexplored scenario is learning with class labels that continually refine/expand old classes. For example, humans learn to recognize ${\tt dog}$ before dog breeds. In practical settings, dataset $\textit{versioning}$ often introduces refinement to ontologies, such as autonomous vehicle benchmarks that refine a previous ${\tt vehicle}$ class into ${\tt school-bus}$ as autonomous operations expand to new cities. This paper formalizes a protocol for studying the problem of $\textit{Learning with Evolving Class Ontology}$ (LECO). LECO requires learning classifiers in distinct time periods (TPs); each TP introduces a new ontology of "fine" labels that refines old ontologies of "coarse" labels (e.g., dog breeds that refine the previous ${\tt dog}$). LECO explores such questions as whether to annotate new data or relabel the old, how to leverage coarse labels, and whether to finetune the previous TP's model or train from scratch. To answer these questions, we leverage insights from related problems such as class-incremental learning. We validate them under the LECO protocol through the lens of image classification (CIFAR and iNaturalist) and semantic segmentation (Mapillary). Our experiments lead to surprising conclusions; while the current status quo is to relabel existing datasets with new ontologies (such as COCO-to-LVIS or Mapillary1.2-to-2.0), LECO demonstrates that a far better strategy is to annotate $\textit{new}$ data with the new ontology. However, this produces an aggregate dataset with inconsistent old-vs-new labels, complicating learning. To address this challenge, we adopt methods from semi-supervised and partial-label learning. Such strategies can surprisingly be made near-optimal, approaching an "oracle" that learns on the aggregate dataset exhaustively labeled with the newest ontology.
DFKI – German Research Center for Artificial Intelligence(德国人工智能研究中心);ZF Friedrichshafen AG(采埃孚腓特烈港股份公司);RPTU – University of Kaiserslautern-Landau(凯泽斯劳滕-兰道莱茵兰-普法尔茨技术大学)
EmbodiedCLBench: Evaluating Continual Learning for Self-Evolving Embodied Agents
School of Intelligence Science and Technology, Peking University(北京大学智能科学与技术学院);State Key Laboratory of General Artificial Intelligence, BIGAI(北京通用人工智能研究院通用人工智能国家重点实验室);University of Hong Kong(香港大学)
An Inclusive and Lightweight Approach to Federated Continual Learning for Cultural Heritage
Information Technologies Institute, Centre for Research and Technology Hellas(希腊研究与技术中心信息技术研究所);School of Electronic Engineering and Computer Science, Queen Mary University of London(伦敦大学玛丽女王学院电子工程与计算机科学学院)
CMKL: Modality-Aware Continual Learning for Evolving Biomedical Knowledge Graphs
Technology, Innovation, Entrepreneurship Department(技术、创新与创业部门);King Abdullah University of Science and Technology(卡塔尔科技大学);School of Computing and Information Systems(计算与信息系统学院);The University of Melbourne(墨尔本大学);College of Computer Science and Technology(计算机科学与技术学院);Jilin University(吉林大学);School of Computing Technologies(计算技术学院);RMIT University(皇家墨尔本理工大学);College of Computing and Data Science(计算与数据科学学院);Nanyang Technological University(南洋理工大学)
From a Prompt to Repertoires: Evolving Functional REpertoires Enable LLM Continual Learning
The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳));Shanghai AI Laboratory(上海人工智能实验室);University of Science and Technology of China(中国科学技术大学);New Jersey Institute of Technology(新泽西理工学院)
Rethinking Continual Experience Internalization for Self-Evolving LLM Agents
Gaoling School of Artificial Intelligence, Renmin University of China(中国人民大学人工智能学院 Gallagher 学院);School of Software, Beihang University(北航软件学院);Meituan(美团)
Semi-parametric Memory Consolidation: Towards Brain-like Deep Continual Learning
Centre for Artificial Intelligence and Robotics(人工智能与机器人中心);Hong Kong Institute of Science and Innovation(香港科学与创新研究院);Chinese Academy of Sciences(中国科学院);Department of Computer Science(计算机科学系);City University of Hong Kong(香港城市大学);The Department of Neurosurgery(神经外科部门);Peking University First Hospital(北京大学第一医院);State Key Laboratory of Multimodal Artificial Intelligence Systems(多模态人工智能系统国家重点实验室);Institute of Automation(自动化研究所);Hong Kong SAR, China(香港特别行政区,中国)
Continual Instruction Tuning for Large Multimodal Models
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);University of Chinese Academy of Sciences(中国科学院大学);The Peng Cheng Laboratory(鹏城实验室);Wuhan AI Research(武汉人工智能研究院)
CoMemNet: A Continual Memory Network with Drift-Aware Sampling for Traffic Prediction
College of Computer Science, Shanghai Jiao Tong University(上海交通大学计算机科学学院);College of Computer Science and Technology, Zhejiang University of Technology(浙江工业大学计算机科学与技术学院);School of Automation, Nanjing University of Science and Technology(南京理工大学自动化学院)
PrimeKG-CL: A Continual Graph Learning Benchmark on Evolving Biomedical Knowledge Graphs
Technology, Innovation, Entrepreneurship Department(技术、创新与创业系);King Abdullah University of Science and Technology(国王阿卜杜勒阿齐兹大学);School of Computing and Information Systems(计算与信息系);The University of Melbourne(墨尔本大学);College of Computer Science and Technology(计算机科学与技术学院);School of Computing Technologies(计算技术学院);Jilin University(吉林大学);RMIT University(皇家墨尔本理工大学);Department of Systems Engineering and Engineering Management(系统工程与工程管理系);The Chinese University of Hong Kong(香港中文大学);Department of Data Science and Artificial Intelligence(数据科学与人工智能系);Hong Kong Polytechnic University(香港理工大学)
Faculty of Information Technology, University of Jyväskylä(约赫斯库利亚大学信息科技学院);National University of Singapore(新加坡国立大学);School of Computer Science and Technology, Dalian University of Technology(大连理工大学计算机科学与技术学院)
SloMo-Fast: Slow-Momentum and Fast-Adaptive Teachers for Source-Free Continual Test-Time Adaptation
Center for Computational & Data Sciences, Independent University, Bangladesh(计算与数据科学中心,独立大学,孟加拉国);University of Central Florida, USA(佛罗里达中央大学,美国);University of Virginia, USA(弗吉尼亚大学,美国);Amazon GenAI, USA(亚马逊生成人工智能,美国)
The Future of Continual Learning in the Era of Foundation Models: Three Key Directions
Department of Computer Science, Università di Pisa(帕尔马大学计算机科学系);Institute of Information Science and Technologies, National Research Council(信息科学与技术研究所,国家研究理事会)
Online Continual Learning with Dynamic Label Hierarchies
College of Computer Science(计算机科学学院);Technology, Nanjing University of Aeronautics(技术学院,南京航空航天大学);MIIT Key Laboratory of Pattern Analysis(模式分析 MIIT 实验室);Computer Vision Center, Spain(西班牙计算机视觉中心);Computer Sciences Department, Universitat Autonoma de Barcelona, Spain(巴塞罗那自治大学计算机科学系,西班牙);State Key Laboratory for Novel Software Technology, Nanjing University(新型软件技术国家重点实验室,南京大学);IDEAS Research Institute, Warsaw, Poland(华沙 IDEAS 研究院,波兰);Joint Laboratory of Spatial intelligent Perception(空间智能感知联合实验室)
Brain-Inspired Hierarchical Modularity for General Continual Learning
Tsinghua University(清华大学);IDG/McGovern Institute for Brain Research, Tsinghua University(清华大学IDG/麦戈文脑科学研究院);Tsinghua-Peking Center for Life Sciences(清华-北大生命科学联合中心)
SOLAR: A Self-Optimizing Open-Ended Autonomous Agent for Lifelong Learning and Continual Adaptation
Department of Ophthalmology, Yong Loo Lin School of Medicine, National University of Singapore, Singapore(眼科学系,Yong Loo Lin医学院,新加坡国立大学,新加坡);Department of Computational and Data Sciences, Indian Institute of Science, Bangalore, Karnataka, India(计算与数据科学系,印度科学研究院,班加罗尔,卡纳塔克邦,印度)
Is Our Benchmark Enough? An Analysis of Continual Learning for MLLMs
School of Computer Science and Statistics, Trinity College Dublin(都柏林圣三一学院计算机科学与统计学院);Department of Mathematics and Computer Science, Eindhoven University of Technology(埃因霍温理工大学数学与计算机科学系)
Adapter-Enhanced Semantic Prompting for Continual Learning
Beijing Key Laboratory of Multimedia and Intelligent Software Technology(北京多媒体与智能软件技术重点实验室);Faculty of Information Technology, Beijing University of Technology(信息科技学院,北京理工大学);University of California at Merced(加州大学默塞德分校);Macquarie University(麦考瑞大学)
Bisecle: Binding and Separation in Continual Learning for Video Language Understanding
School of Computer Science University of New South Wales(计算机科学学院新南威尔士大学);School of Computer Science and Engineering University of New South Wales(计算机科学与工程学院新南威尔士大学);DEVCOM Army Research Laboratory(陆军研究实验室)