Catastrophic forgetting makes neural network models unstable when learning visual domains consecutively. The neural network model drifts to catastrophic forgetting-induced low performance of previously learnt domains when training with new domains. We illuminate this current neural network model weakness and develop a forgetting-resistant incremental learning strategy. Here, we propose a new unsupervised incremental open-set domain adaptation (IOSDA) issue for image classification. Open-set domain adaptation adds complexity to the incremental domain adaptation issue since each target domain has more classes than the Source domain. In IOSDA, the model learns training with domain streams phase by phase in incremented time. Inference uses test data from all target domains without revealing their identities. We proposed IOSDA-Net, a two-stage learning pipeline, to solve the problem. The first module replicates prior domains from random noise using a generative framework and creates a pseudo source domain. In the second step, this pseudo source is adapted to the present target domain. We test our model on Office-Home, DomainNet, and UPRN-RSDA, a newly curated optical remote sensing dataset.
Peking University（北京大学）；South China University of Technology（华南理工大学）；National University of Singapore（新加坡国立大学）；MIT-IBM Watson AI Lab（麻省理工学院-IBM沃森人工智能实验室）
This paper studies a new, practical but challenging problem, called Class-Incremental Unsupervised Domain Adaptation (CI-UDA), where the labeled source domain contains all classes, but the classes in the unlabeled target domain increase sequentially. This problem is challenging due to two difficulties. First, source and target label sets are inconsistent at each time step, which makes it difficult to conduct accurate domain alignment. Second, previous target classes are unavailable in the current step, resulting in the forgetting of previous knowledge. To address this problem, we propose a nov…
École de technologie supérieure（高等技术学院）；Genetec Inc.（捷泰科公司）
Recent advances in unsupervised domain adaptation have significantly improved the recognition accuracy of CNNs by alleviating the domain shift between (labeled) source and (unlabeled) target data distributions. While the problem of single-target domain adaptation (STDA) for object detection has recently received much attention, multi-target domain adaptation (MTDA) remains largely unexplored, despite its practical relevance in several real-world applications, such as multi-camera video surveillance. Compared to the STDA problem that may involve large domain shifts between complex source and ta…
Multi-Granularity Class Prototype Topology Distillation for Class-Incremental Source-Free Unsupervised Domain Adaptation
Hangzhou Dianzi University（杭州电子科技大学）；Macao Polytechnic University（澳门理工大学）；Beijing Institute of Technology（北京理工大学）；Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）
Indian Institute of Technology Bombay（印度理工学院孟买分校）；Goethe University Frankfurt（法兰克福歌德大学）
We tackle the novel problem of incremental unsupervised domain adaptation (IDA) in this paper. We assume that a labeled source domain and different unlabeled target domains are incrementally observed with the constraint that data corresponding to the current domain is only available at a time. The goal is to preserve the accuracies for all the past domains while generalizing well for the current domain. The IDA setup suffers due to the abrupt differences among the domains and the unavailability of past data including the source domain. Inspired by the notion of generative feature replay, we pr…
We introduce a practical Domain Adaptation (DA) paradigm called Class-Incremental Domain Adaptation (CIDA). Existing DA methods tackle domain-shift but are unsuitable for learning novel target-domain classes. Meanwhile, class-incremental (CI) methods enable learning of new classes in absence of source training data but fail under a domain-shift without labeled supervision. In this work, we effectively identify the limitations of these approaches in the CIDA paradigm. Motivated by theoretical and empirical observations, we propose an effective method, inspired by prototypical networks, that ena…
Continuous appearance shifts such as changes in weather and lighting conditions can impact the performance of deployed machine learning models. While unsupervised domain adaptation aims to address this challenge, current approaches do not utilise the continuity of the occurring shifts. In particular, many robotics applications exhibit these conditions and thus facilitate the potential to incrementally adapt a learnt model over minor shifts which integrate to massive differences over time. Our work presents an adversarial approach for lifelong, incremental domain adaptation which benefits from …
Khalifa University（哈利法大学）；National University of Sciences and Technology（国家科技大学）；Beihang University（北京航空航天大学）；Mayo Clinic（梅奥诊所）；Abu Dhabi Healthcare Company (SEHA)（阿布扎比医疗公司（SEHA））
Retinopathy represents a group of retinal diseases that, if not treated timely, can cause severe visual impairments or even blindness. Many researchers have developed autonomous systems to recognize retinopathy via fundus and optical coherence tomography (OCT) imagery. However, most of these frameworks employ conventional transfer learning and fine-tuning approaches, requiring a decent amount of well-annotated training data to produce accurate diagnostic performance. This paper presents a novel incremental cross-domain adaptation instrument that allows any deep classification model to progress…
National University of Singapore（新加坡国立大学）；NUSRI Suzhou（新加坡国立大学苏州研究院）；Singapore General Hospital（新加坡中央医院）；The Chinese University of Hong Kong（香港中文大学）；Imperial College London（伦敦帝国理工学院）
Generating surgical reports aimed at surgical scene understanding in robot-assisted surgery can contribute to documenting entry tasks and post-operative analysis. Despite the impressive outcome, the deep learning model degrades the performance when applied to different domains encountering domain shifts. In addition, there are new instruments and variations in surgical tissues appeared in robotic surgery. In this work, we propose class-incremental domain adaptation (CIDA) with a multi-layer transformer-based model to tackle the new classes and domain shift in the target domain to generate surg…
L3A: Label-Augmented Analytic Adaptation for Multi-Label Class Incremental Learning
Shien-Ming Wu School of Intelligent Engineering, South China University of Technology, Guangzhou, China（华南理工大学智能工程学院）；School of Future Technology, South China University of Technology, Guangzhou, China（华南理工大学未来技术学院）；Guangdong Laboratory of Artificial Intelligence（人工智能与数字经济广东省实验室）
Cross-Domain Attribute Alignment with CLIP: A Rehearsal-Free Approach for Class-Incremental Unsupervised Domain Adaptation
School of Computer Science and Engineering（计算机科学与工程学院）；Nanjing University of Science and Technology（南京理工大学）；School of Automation Science and Electrical Engineering（自动化科学与电气工程学院）；Beihang University（北航）；School of Automation and Intelligent Sensing（自动化与智能感知学院）；Shanghai Jiao Tong University（上海交通大学）
Porsche Engineering Services GmbH（保时捷工程服务有限公司）；Fraunhofer IOSB（弗劳恩霍夫光学、系统技术与图像利用研究所）；Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院）
The field of continual deep learning is an emerging field and a lot of progress has been made. However, concurrently most of the approaches are only tested on the task of image classification, which is not relevant in the field of intelligent vehicles. Only recently approaches for class-incremental semantic segmentation were proposed. However, all of those approaches are based on some form of knowledge distillation. At the moment there are no investigations on replay-based approaches that are commonly used for object recognition in a continual setting. At the same time while unsupervised domai…
University of Alicante（阿利坎特大学）；University of Edinburgh（爱丁堡大学）
In the context of supervised statistical learning, it is typically assumed that the training set comes from the same distribution that draws the test samples. When this is not the case, the behavior of the learned model is unpredictable and becomes dependent upon the degree of similarity between the distribution of the training set and the distribution of the test set. One of the research topics that investigates this scenario is referred to as domain adaptation. Deep neural networks brought dramatic advances in pattern recognition and that is why there have been many attempts to provide good …
Advancing Cross-domain Discriminability in Continual Learning of Vision-Language Models
Institute of Science Tokyo（东京科学研究院）；University of California Berkeley（加州大学伯克利分校）；Nanyang Technological University（南洋理工大学）；South China University of Technology（华南理工大学）；Greater Bay Area Institute for Innovation, Hunan University（湖南大学粤港澳大湾区创新研究院）
ShanghaiTech University（上海科技大学）；Shanghai Institute of Microsystem and Information Technology, Chinese Academy of Sciences（中国科学院上海微系统与信息技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Shanghai Engineering Research Center of Intelligent Vision and Imaging（上海智能视觉成像工程技术研究中心）
Continual learning is an important problem for achieving human-level intelligence in real-world applications as an agent must continuously accumulate knowledge in response to streaming data/tasks. In this work, we consider a general and yet under-explored incremental learning problem in which both the class distribution and class-specific domain distribution change over time. In addition to the typical challenges in class incremental learning, this setting also faces the intra-class stability-plasticity dilemma and intra-class domain imbalance problems. To address above issues, we develop a no…
This notebook paper presents an overview and comparative analysis of our systems designed for the following two tasks in Visual Domain Adaptation Challenge (VisDA-2019): multi-source domain adaptation and semi-supervised domain adaptation. Multi-Source Domain Adaptation: We investigate both pixel-level and feature-level adaptation for multi-source domain adaptation task, i.e., directly hallucinating labeled target sample via CycleGAN and learning domain-invariant feature representations through self-learning. Moreover, the mechanism of fusing features from different backbones is further studie…
Few-shot Class-Incremental Learning via Generative Co-Memory Regularization
Institute of Information Engineering（信息工程研究所）；Chinese Academy of Sciences（中国科学院）；School of Cyber Security（网络安全学院）；University of Chinese Academy of Sciences（中国科学院大学）；Department of Communication Engineering（通信工程系）
A Survey of Incremental Transfer Learning: Combining Peer-to-Peer Federated Learning and Domain Incremental Learning for Multicenter Collaboration
University Hospital Erlangen（埃尔朗根大学医院）；Friedrich-Alexander-Universität Erlangen-Nürnberg（埃尔朗根-纽伦堡弗里德里希-亚历山大大学）；Comprehensive Cancer Center Erlangen-EMN（埃尔朗根-EMN综合癌症中心）；Bavarian Cancer Research Center（巴伐利亚癌症研究中心）
Deep learning has achieved notable success in 3D object detection with the advent of large-scale point cloud datasets. However, severe performance degradation in the past trained classes, i.e., catastrophic forgetting, still remains a critical issue for real-world deployment when the number of classes is unknown or may vary. Moreover, existing 3D class-incremental detection methods are developed for the single-domain scenario, which fail when encountering domain shift caused by different datasets, varying environments, etc. In this paper, we identify the unexplored yet valuable scenario, i.e.,…
Enhancing Plasticity for First Session Adaptation Continual Learning
LTCI, Télécom-Paris, Institut Polytechnique de Paris（Télécom-Paris 工程院）；University of Bergamo（贝加莫大学）；Inria at University Grenoble Alpes, LJK（格勒诺布尔阿尔卑斯大学）
Traditional learning systems are trained in closed-world for a fixed number of classes, and need pre-collected datasets in advance. However, new classes often emerge in real-world applications and should be learned incrementally. For example, in electronic commerce, new types of products appear daily, and in a social media community, new topics emerge frequently. Under such circumstances, incremental models should learn several new classes at a time without forgetting. We find a strong correlation between old and new classes in incremental learning, which can be applied to relate and facilitat…
Fudan University（复旦大学）；Xiamen University（厦门大学）；The Pennsylvania State University（宾夕法尼亚州立大学）；Chinese Academy of Sciences（中国科学院）
Crowd counting has important applications in public safety and pandemic control. A robust and practical crowd counting system has to be capable of continuously learning with the new incoming domain data in real-world scenarios instead of fitting one domain only. Off-the-shelf methods have some drawbacks when handling multiple domains: (1) the models will achieve limited performance (even drop dramatically) among old domains after training images from new domains due to the discrepancies of intrinsic data distributions from various domains, which is called catastrophic forgetting; (2) the well-…
University of Cambridge（剑桥大学）；Binghamton University（宾汉姆顿大学）；Beijing University of Posts and Telecommunications（北京邮电大学）
As Facial Expression Recognition (FER) systems become integrated into our daily lives, these systems need to prioritise making fair decisions instead of aiming at higher individual accuracy scores. Ranging from surveillance systems to diagnosing mental and emotional health conditions of individuals, these systems need to balance the accuracy vs fairness trade-off to make decisions that do not unjustly discriminate against specific under-represented demographic groups. Identifying bias as a critical problem in facial analysis systems, different methods have been proposed that aim to mitigate bi…
Google Research（谷歌研究院）；University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
We introduce an approach for incremental learning that preserves feature descriptors of training images from previously learned classes, instead of the images themselves, unlike most existing work. Keeping the much lower-dimensional feature embeddings of images reduces the memory footprint significantly. We assume that the model is updated incrementally for new classes as new data becomes available sequentially.This requires adapting the previously stored feature vectors to the updated feature space without having access to the corresponding original training images. Feature adaptation is lear…
Mamba-FSCIL: Dynamic Adaptation with Selective State Space Model for Few-Shot Class-Incremental Learning
School of Computer Science and Technology, Harbin Institute of Technology (Shenzhen)（哈尔滨工业大学（深圳）计算机科学与技术学院）；Pengcheng Laboratory（鹏城实验室）；King Abdullah University of Science and Technology（国王阿卜杜勒-阿齐兹大学）；Department of Computer Science and Engineering, University of California, Merced（加州大学默塞德分校计算机科学与工程系）
Exploiting Stability-Plasticity Asymmetry in Pretrained Detectors for Incremental Object Detection
Harbin Institute of Technology（哈尔滨工业大学）；State Key Laboratory of Multimodal Artificial Intelligence Systems, CASIA（中国科学院自动化研究所多模态人工智能系统国家重点实验室）；School of Artificial Intelligence, UCAS（中国科学院大学人工智能学院）
Recent efforts in multi-domain learning for semantic segmentation attempt to learn multiple geographical datasets in a universal, joint model. A simple fine-tuning experiment performed sequentially on three popular road scene segmentation datasets demonstrates that existing segmentation frameworks fail at incrementally learning on a series of visually disparate geographical domains. When learning a new domain, the model catastrophically forgets previously learned knowledge. In this work, we pose the problem of multi-domain incremental learning for semantic segmentation. Given a model trained o…
Given an existing trained neural network, it is often desirable to learn new capabilities without hindering performance of those already learned. Existing approaches either learn sub-optimal solutions, require joint training, or incur a substantial increment in the number of parameters for each added domain, typically as many as the original network. We propose a method called \emph{Deep Adaptation Networks} (DAN) that constrains newly learned filters to be linear combinations of existing ones. DANs precisely preserve performance on the original domain, require a fraction (typically 13\%, depe…
Computer Vision Center UAB（巴塞罗那自治大学计算机视觉中心）；Amazon Web Services（亚马逊云科技）；UCLA（加州大学洛杉矶分校）
Multi-task learns multiple tasks, while sharing knowledge and computation among them. However, it suffers from catastrophic forgetting of previous knowledge when learned incrementally without access to the old data. Most existing object detectors are domain-specific and static, while some are learned incrementally but only within a single domain. Training an object detector incrementally across various domains has rarely been explored. In this work, we propose three incremental learning scenarios across various domains and categories for object detection. To mitigate catastrophic forgetting, a…
The Chinese University of Hong Kong（香港中文大学）；The University of Hong Kong（香港大学）
Contemporary methods have shown promising results on cardiac image segmentation, but merely in static learning, i.e., optimizing the network once for all, ignoring potential needs for model updating. In real-world scenarios, new data continues to be gathered from multiple institutions over time and new demands keep growing to pursue more satisfying performance. The desired model should incrementally learn from each incoming dataset and progressively update with improved functionality as time goes by. As the datasets sequentially delivered from multiple sites are normally heterogenous with doma…
Zhejiang University（浙江大学）；Zhejiang Lab（之江实验室）；University of California, Los Angeles（加州大学洛杉矶分校）
Conventional detection networks usually need abundant labeled training samples, while humans can learn new concepts incrementally with just a few examples. This paper focuses on a more challenging but realistic class-incremental few-shot object detection problem (iFSD). It aims to incrementally transfer the model for novel objects from only a few annotated samples without catastrophically forgetting the previously learned ones. To tackle this problem, we propose a novel method LEAST, which can transfer with Less forgetting, fEwer training resources, And Stronger Transfer capability. Specifical…
