Multi-task Learning (MTL) for classification with disjoint datasets aims to explore MTL when one task only has one labeled dataset. In existing methods, for each task, the unlabeled datasets are not fully exploited to facilitate this task. Inspired by semi-supervised learning, we use unlabeled datasets with pseudo labels to facilitate each task. However, there are two major issues: 1) the pseudo labels are very noisy; 2) the unlabeled datasets and the labeled dataset for each task has considerable data distribution mismatch. To address these issues, we propose our MTL with Selective Augmentation (MTL-SA) method to select the training samples in unlabeled datasets with confident pseudo labels and close data distribution to the labeled dataset. Then, we use the selected training samples to add information and use the remaining training samples to preserve information. Extensive experiments on face-centric and human-centric applications demonstrate the effectiveness of our MTL-SA method.
Facebook AI Research（Facebook 人工智能研究院）；UC Berkeley EECS（加州大学伯克利分校电气工程与计算机科学系）
Continual learning aims to learn new tasks without forgetting previously learned ones. We hypothesize that representations learned to solve each task in a sequence have a shared structure while containing some task-specific properties. We show that shared features are significantly less prone to forgetting and propose a novel hybrid continual learning framework that learns a disjoint representation for task-invariant and task-specific features required to solve a sequence of tasks. Our model combines architecture growth to prevent forgetting of task-specific skills and an experience replay app…
Warsaw University of Technology（华沙理工大学）；Jagiellonian University（雅盖隆大学）；Tooploox
Catastrophic forgetting of previously learned knowledge while learning new tasks is a widely observed limitation of contemporary neural networks. Although many continual learning methods are proposed to mitigate this drawback, the main question remains unanswered: what is the root cause of catastrophic forgetting? In this work, we aim at answering this question by posing and validating a set of research hypotheses related to the specificity of representations built internally by neural models. More specifically, we design a set of empirical evaluations that compare the robustness of representa…
The Australian National University（澳大利亚国立大学）；NEC Labs America（美国NEC实验室）；Monash University（莫纳什大学）；Data61（Data61研究院）；Phiar Technologies（Phiar科技公司）；University of California, San Diego（加州大学圣地亚哥分校）
Humans have the ability to accumulate knowledge of new tasks in varying conditions, but deep neural networks often suffer from catastrophic forgetting of previously learned knowledge after learning a new task. Many recent methods focus on preventing catastrophic forgetting under the assumption of train and test data following similar distributions. In this work, we consider a more realistic scenario of continual learning under domain shifts where the model must generalize its inference to an unseen domain. To this end, we encourage learning semantically meaningful features by equipping the cla…
University of Florence（佛罗伦萨大学）；Universitat Autónoma de Barcelona（巴塞罗那自治大学）
Research on continual learning has led to a variety of approaches to mitigating catastrophic forgetting in feed-forward classification networks. Until now surprisingly little attention has been focused on continual learning of recurrent models applied to problems like image captioning. In this paper we take a systematic look at continual learning of LSTM-based models for image captioning. We propose an attention-based approach that explicitly accommodates the transient nature of vocabularies in continual image captioning tasks -- i.e. that task vocabularies are not disjoint. We call our method…
Cross-Domain Attribute Alignment with CLIP: A Rehearsal-Free Approach for Class-Incremental Unsupervised Domain Adaptation
School of Computer Science and Engineering（计算机科学与工程学院）；Nanjing University of Science and Technology（南京理工大学）；School of Automation Science and Electrical Engineering（自动化科学与电气工程学院）；Beihang University（北航）；School of Automation and Intelligent Sensing（自动化与智能感知学院）；Shanghai Jiao Tong University（上海交通大学）
Unveiling the Tapestry: the Interplay of Generalization and Forgetting in Continual Learning
Nanyang Technological University（南洋理工大学）；Agency for Science, Technology and Research（科学、技术与研究局）；Hefei University of Technology（合肥工业大学）；Sichuan University（四川大学）
A practical eye authentication (EA) system targeted for edge devices needs to perform authentication and be robust to presentation attacks, all while remaining compute and latency efficient. However, existing eye-based frameworks a) perform authentication and Presentation Attack Detection (PAD) independently and b) involve significant pre-processing steps to extract the iris region. Here, we introduce a joint framework for EA and PAD using periocular images. While a deep Multitask Learning (MTL) network can perform both the tasks, MTL suffers from the forgetting effect since the training datas…
Statistically Meaningful Geometry (SMG) Beyond the Euclidean Paradigm, with Application to Generative AI
Academy of Mathematics and Systems Science, Chinese Academy of Sciences（中国科学院数学与系统科学研究院）；AMSS Center for Forecasting Science, Chinese Academy of Sciences（中国科学院数学与系统科学研究院预测科学研究中心）；Beijing Institute of Mathematical Sciences and Applications (BIMSA)（北京国际数学研究中心）；Department of Statistics, London School of Economics and Political Science（伦敦政治经济学院统计系）；Department of Statistics and Data Science, Tsinghua University（清华大学统计与数据科学系）；Paula and Gregory Chow Institute for the Studies in Economics, Xiamen University（厦门大学经济研究所（保拉和格雷戈里·周研究所））；Yau Mathematical Sciences Center, Tsinghua University（清华大学丘成桐数学科学中心）
Knowing Beyond the Known: Reinforced Knowledge Specification for Multi-Label Class-Incremental Learning
Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；School of Cyber Security, University of Chinese Academy of Sciences（中国科学院大学网络空间安全学院）；Nankai University（南开大学）；Tsinghua University（清华大学）；Harbin Institute of Technology（哈尔滨工业大学）
