Continual learning aims to refine model parameters for new tasks while retaining knowledge from previous tasks. Recently, prompt-based learning has emerged to leverage pre-trained models to be prompted to learn subsequent tasks without the reliance on the rehearsal buffer. Although this approach has demonstrated outstanding results, existing methods depend on preceding task-selection process to choose appropriate prompts. However, imperfectness in task-selection may lead to negative impacts on the performance particularly in the scenarios where the number of tasks is large or task distributions are imbalanced. To address this issue, we introduce I-Prompt, a task-agnostic approach focuses on the visual semantic information of image tokens to eliminate task prediction. Our method consists of semantic prompt matching, which determines prompts based on similarities between tokens, and image token-level prompting, which applies prompts directly to image tokens in the intermediate layers. Consequently, our method achieves competitive performance on four benchmarks while significantly reducing training time compared to state-of-the-art methods. Moreover, we demonstrate the superiority of our method across various scenarios through extensive experiments.
Consistent Prompting for Rehearsal-Free Continual Learning
School of Artificial Intelligence, Sun Yat-sen University(中山大学人工智能学院);Cheng Kar-Shun Robotics Institute, The Hong Kong University of Science and Technology(香港科技大学郑家纯机器人研究院);Key Laboratory of Machine Intelligence and Advanced Computing, Ministry of Education(教育部机器智能与先进计算重点实验室)
Adapter-Enhanced Semantic Prompting for Continual Learning
Beijing Key Laboratory of Multimedia and Intelligent Software Technology(北京多媒体与智能软件技术重点实验室);Faculty of Information Technology, Beijing University of Technology(信息科技学院,北京理工大学);University of California at Merced(加州大学默塞德分校);Macquarie University(麦考瑞大学)
Pontificia Universidad Católica de Chile(智利天主教大学);King Abdullah University of Science and Technology (KAUST)(阿卜杜拉国王科技大学);University of Pisa(比萨大学);Adobe Research(Adobe研究院)
Critical Patch-Aware Sparse Prompting with Decoupled Training for Continual Learning on the Edge
School of Computer Science and Engineering, Chung-Ang University(中央大学计算机科学与工程学院);Department of Artificial Intelligence, Chung-Ang University(中央大学人工智能系)
GAP-Prompt: Gated Adaptive Prompting for Efficient Continual Learning
Université d’Orléans(奥尔良大学);INSA CVL(中央卢瓦尔河谷国立应用科学学院);CY Cergy Paris University(塞尔吉-蓬图瓦兹大学);ENSEA(法国高等电子与数字学院);CNRS(法国国家科学研究中心);Université de Technologie de Troyes(特鲁瓦技术大学)
Dynamically Anchored Prompting for Task-Imbalanced Continual Learning
Xiamen University(厦门大学);Univ. Grenoble Alpes(格勒诺布尔大学);Inria(法国国家信息与自动化研究所);CNRS(法国国家科学研究中心);Grenoble INP(格勒诺布尔理工学院);LJK(让·昆廷诺实验室);Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ)(广东省人工智能与数字经济实验室(深圳))
PM2: A New Prompting Multi-modal Model Paradigm for Few-shot Medical Image Classification
Dalian University of Technology(大连理工大学);School of Computer Science and Technology(计算机科学与技术学院);School of Computer Science and Engineering(计算机科学与工程学院);Dalian Minzu University(大连民族大学);School of Information and Communication Engineering(信息与通信工程学院)
Pengcheng laboratory(鹏城实验室);Harbin Institute of Technology(哈尔滨工业大学);Xi’an Jiaotong University(西安交通大学);Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences(中国科学院深圳先进技术研究院)
CM2-Net: Continual Cross-Modal Mapping Network for Driver Action Recognition
School of Electrical and Electronic Engineering, Nanyang Technological University(南洋理工大学电气与电子工程学院);Continental-NTU Corporate Lab, Nanyang Technological University(大陆集团-南洋理工大学联合企业实验室);College of Computer Science and Technology, Harbin Engineering University(哈尔滨工程大学计算机科学与技术学院)
Continual Learning for Segment Anything Model Adaptation
Southern University of Science and Technology(南方科技大学);City University of Hong Kong(香港城市大学);Hong Kong University of Science and Technology(香港科技大学);Tencent YouTu Lab(腾讯优图实验室)
针对动态流式数据场景,提出持续 SAM 适应基准 CoSAM 和域适配器混合算法 MoDA,通过全局特征与辅助令牌提取分离特征,实现持续分割并保持知识,优于现有方法。
Continual Test-Time Adaptation in Computer Vision: Methods, Benchmarks, and Future Directions
The University of Texas at Dallas(德克萨斯大学达拉斯分校);LIVIA ETS Montreal, ILLS International Laboratory on Learning Systems (ILLS)(蒙特利尔LIVIA ETS,学习系统国际实验室(ILLS));Tulane University(路易斯安那州立大学);Nanyang Technological University(南洋理工大学);Hanyang University(翰阳大学)
Continual-MAE: Adaptive Distribution Masked Autoencoders for Continual Test-Time Adaptation
National Key Laboratory for Multimedia Information Processing(多媒体信息处理国家重点实验室);School of Computer Science(计算机学院);Peking University(北京大学);CUHK(香港中文大学);University of Science Technology of China(中国科学技术大学)
Hierarchical-Task-Aware Multi-modal Mixture of Incremental LoRA Experts for Embodied Continual Learning
Ping An Technology (Shenzhen) Co., Ltd.(平安科技(深圳)有限公司);Tsinghua Shenzhen International Graduate School(清华大学深圳国际研究生院);Tsinghua University(清华大学);Huazhong University of Science and Technology(华中科技大学)
What to Remember: Self-Adaptive Continual Learning for Audio Deepfake Detection
State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所多模态人工智能系统国家重点实验室);School of Computer and Information Technology, University of Beijing Jiaotong(北京交通大学计算机与信息技术学院);Department of Automation, Tsinghua University(清华大学自动化系);University of Science and Technology of China(中国科学技术大学)
CRISP: Contrastive Residual Injection and Semantic Prompting for Continual Video Instance Segmentation
State Key Laboratory of Robotics and Intelligent Systems, Shenyang Institute of Automation, Chinese Academy of Sciences(机器人与智能系统国家重点实验室,沈阳自动化研究所,中国科学院);University of Chinese Academy of Sciences(中国科学院大学);Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学)
Harbin Institute of Technology, Shenzhen(哈尔滨工业大学(深圳));vivo Mobile Communication Co., Ltd.(维沃移动通信有限公司);Shenyang Institute of Automation, Chinese Academy of Sciences(中国科学院沈阳自动化研究所);Pengcheng Laboratory(鹏城实验室)