Although existing cross-domain continual learning approaches successfully address many streaming tasks having domain shifts, they call for a fully labeled source domain hindering their feasibility in the privacy constrained environments. This paper goes one step ahead with the problem of source-free cross-domain continual learning where the use of source-domain samples are completely prohibited. We propose the idea of rehearsal-free frequency-aware dynamic prompt collaborations (REFEREE) to cope with the absence of labeled source-domain samples in realm of cross-domain continual learning. REFEREE is built upon a synergy between a source-pre-trained model and a large-scale vision-language model, thus overcoming the problem of sub-optimal generalizations when relying only on a source pre-trained model. The domain shift problem between the source domain and the target domain is handled by a frequency-aware prompting technique encouraging low-frequency components while suppressing high-frequency components. This strategy generates frequency-aware augmented samples, robust against noisy pseudo labels. The noisy pseudo-label problem is further addressed with the uncertainty-aware weighting strategy where the mean and covariance matrix are weighted by prediction uncertainties, thus mitigating the adverse effects of the noisy pseudo label. Besides, the issue of catastrophic forgetting (CF) is overcome by kernel linear discriminant analysis (KLDA) where the backbone network is frozen while the classification is performed using the linear discriminant analysis approach guided by the random kernel method. Our rigorous numerical studies confirm the advantage of our approach where it beats prior arts having access to source domain samples with significant margins.
Prototype-Guided Non-Exemplar Continual Learning for Cross-subject EEG Decoding
Dept. of Artificial Intelligence Korea University Seoul, Republic of Korea(人工智能系 韩国大学首尔共和国);Cognitive Engineering Korea University Seoul, Republic of Korea(认知工程 韩国大学首尔共和国)
MM-Prompt: Cross-Modal Prompt Tuning for Continual Visual Question Answering
Khoury College of Computer Sciences, Northeastern University(东北大学克劳尔计算机科学学院);New Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所模式识别新实验室)
Dynamic Cross-Modal Prompt Generation for Multimodal Continual Instruction Tuning
School of Artificial Intelligence, Nanjing University(南京大学人工智能学院);State Key Laboratory for Novel Software Technology, Nanjing University(南京大学新型软件技术国家重点实验室)
Nanyang Technological University(南洋理工大学);University of South Australia(南澳大学);Singapore Institute of Manufacturing Technology (SIMTech)(新加坡制造技术研究院);Agency for Science, Technology and Research (A*STAR)(新加坡科技研究局)
Exploring Fine-tuning Techniques for Pre-trained Cross-lingual Models via Continual Learning
Center for Artificial Intelligence Research (CAiRE)(人工智能研究中心(CAiRE));Department of Electronic and Computer Engineering(电子与计算机工程系);The Hong Kong University of Science and Technology(香港科技大学)
Rehearsal-Free Modular and Compositional Continual Learning for Language Models
Bosch Center for Artificial Intelligence(博世人工智能中心);LMU Munich(慕尼黑大学);Hochschule der Medien(斯图加特媒体大学);Karlsruhe University of Applied Sciences(卡尔斯鲁厄应用技术大学)
Subspace Optimization for Backpropagation-Free Continual Test-Time Adaptation
Poznan University of Technology(波兹南理工大学);IDEAS NCBR;NASK - National Research Institute(NASK国家研究所);Gdańsk University of Technology(格但斯克理工大学);Graz University of Technology(格拉茨理工大学)
Prior-free Balanced Replay: Uncertainty-guided Reservoir Sampling for Long-Tailed Continual Learning
Shenzhen Research Institute of Big Data (SRIBD)(深圳大数据研究院);The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳));The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));The Hong Kong University of Science and Technology(香港科技大学);The Hong Kong Polytechnic University(香港理工大学)
The Chinese University of Hong Kong(香港中文大学);National University of Singapore(新加坡国立大学);Yale University(耶鲁大学);The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州))
CM2-Net: Continual Cross-Modal Mapping Network for Driver Action Recognition
School of Electrical and Electronic Engineering, Nanyang Technological University(南洋理工大学电气与电子工程学院);Continental-NTU Corporate Lab, Nanyang Technological University(大陆集团-南洋理工大学联合企业实验室);College of Computer Science and Technology, Harbin Engineering University(哈尔滨工程大学计算机科学与技术学院)
Octopus: History-Free Gradient Orthogonalization for Continual Learning in Multimodal Large Language Models
MoE Key Lab of Artificial Intelligence, AI Institute, Shanghai Jiao Tong University(人工智能大模型关键实验室,人工智能研究院,上海交通大学);vivo Mobile Communication Co., Ltd.(vivo移动通信有限公司)
On Generalizing Beyond Domains in Cross-Domain Continual Learning
The Australian National University(澳大利亚国立大学);NEC Labs America(美国NEC实验室);Monash University(莫纳什大学);Data61(Data61研究院);Phiar Technologies(Phiar科技公司);University of California, San Diego(加州大学圣地亚哥分校)
FreeMOCA: Memory-Free Continual Learning for Malicious Code Analysis
Department of Computer Engineering(计算机工程系);Amirkabir University of Technology(阿姆irkabir技术大学);Division of Artificial Intelligence & Software(人工智能与软件系);Ewha Womans University(成均馆大学);Department of Computer Science(计算机科学系);University of Texas at El Paso(德克萨斯理工大学)
PROL : Rehearsal Free Continual Learning in Streaming Data via Prompt Online Learning
STEM University of South Australia(南澳大利亚STEM大学);Institute for Infocomm Research, A*STAR & IPAL(信息与通信研究所,A*STAR及IPAL);Systems Research Institute, Polish Academy of Sciences(波兰科学院系统研究所)
Forget-free Continual Learning with Soft-Winning SubNetworks
KAIST(韩国科学技术院)
受正则化彩票假说(RLTH)的启发,该假说指出,竞争性光滑(非二进制)子网络存在于一个密集的网络中的持续学习任务,我们研究了两个建议的基于架构的持续学习方法,顺序学习和选择自适应二进制(WSN)和非二进制软子网(SoftNet)为每个任务。WSN和SoftNet共同学习与每个任务相关联的子网络的正则化模型权重和任务自适应非二进制掩码,同时尝试通过重用先前子网络的权重来选择要激活的一小组权重(中奖票)。我们提出的无线传感器网络和软网络是固有的免疫灾难性遗忘,因为每个选定的子网络模型不侵犯其他子网络的任务增量学习(TIL)。在TIL中,每个中奖彩票产生的二进制掩码被编码成一个N位二进制数字掩码,然后使用霍夫曼编码进行压缩,以使网络容量与任务数量呈次线性增长。令人惊讶的是,在推理步骤中,通过向所获取的WSN的背景注入小噪声(保持WSN的前景)而生成的软网为TIL中的未来任务提供了出色的前向传输能力。SoftNet在Few-Shot Class Incremental Learning(FSCIL)中的一些示例中显示了其在正则化参数以解决过拟合方面优于WSN的有效性。