测试时自适应（Test-Time Adaptation, TTA）针对未见过的测试数据微调预训练深度神经网络。TTA的主要挑战在于在线更新期间无法访问完整测试数据集，这会导致误差累积。为缓解该问题，现有TTA方法将模型输出的熵用作置信度指标，旨在判定哪些样本引发误差的可能性更低。然而，通过实验研究我们观察到，在有偏场景下，熵作为TTA的置信度指标并不可靠；我们还从理论上揭示，这一问题的根源在于忽略了数据的潜在解耦因子对预测结果的影响。基于这些发现，我们提出了一种名为Destroy Your Object（DeYO）的新型TTA方法，该方法采用了我们新提出的置信度指标——伪标签概率差（Pseudo-Label Probability Difference, PLPD）。PLPD通过测量施加对象破坏变换前后的预测结果差异，量化对象形状对预测的影响。DeYO包含样本选择和样本加权两个模块，同时结合使用熵和PLPD。为实现鲁棒自适应，DeYO会优先选择预测时主要依赖形状信息的样本。我们的大量实验表明，在包括有偏场景和真实野外场景在内的多种场景下，DeYO的性能始终优于基线方法。项目页面公开于https://whitesnowdrop.github.io/DeYO/。
英文摘要：
Test-time adaptation (TTA) fine-tunes pre-trained deep neural networks for unseen test data. The primary challenge of TTA is limited access to the entire test dataset during online updates, causing error accumulation. To mitigate it, TTA methods have utilized the model output's entropy as a confidence metric that aims to determine which samples have a lower likelihood of causing error. Through experimental studies, however, we observed the unreliability of entropy as a confidence metric for TTA under biased scenarios and theoretically revealed that it stems from the neglect of the influence of latent disentangled factors of data on predictions. Building upon these findings, we introduce a novel TTA method named Destroy Your Object (DeYO), which leverages a newly proposed confidence metric named Pseudo-Label Probability Difference (PLPD). PLPD quantifies the influence of the shape of an object on prediction by measuring the difference between predictions before and after applying an object-destructive transformation. DeYO consists of sample selection and sample weighting, which employ entropy and PLPD concurrently. For robust adaptation, DeYO prioritizes samples that dominantly incorporate shape information when making predictions. Our extensive experiments demonstrate the consistent superiority of DeYO over baseline methods across various scenarios, including biased and wild. Project page is publicly available at https://whitesnowdrop.github.io/DeYO/.
Guangdong University of Technology（广东工业大学）；Nanyang Technological University（南洋理工大学）；Carnegie Mellon University（卡内基梅隆大学）；Foshan University（佛山科学技术学院）
Domain adaptation is an important but challenging task. Most of the existing domain adaptation methods struggle to extract the domain-invariant representation on the feature space with entangling domain information and semantic information. Different from previous efforts on the entangled feature space, we aim to extract the domain invariant semantic information in the latent disentangled semantic representation (DSR) of the data. In DSR, we assume the data generation process is controlled by two independent sets of variables, i.e., the semantic latent variables and the domain latent variables…
University of New South Wales（新南威尔士大学）；National Acoustic Laboratories（国家声学实验室）；Macquarie University（麦考瑞大学）
EEG-based tinnitus classification is a valuable tool for tinnitus diagnosis, research, and treatments. Most current works are limited to a single dataset where data patterns are similar. But EEG signals are highly non-stationary, resulting in model's poor generalization to new users, sessions or datasets. Thus, designing a model that can generalize to new datasets is beneficial and indispensable. To mitigate distribution discrepancy across datasets, we propose to achieve Disentangled and Side-aware Unsupervised Domain Adaptation (DSUDA) for cross-dataset tinnitus diagnosis. A disentangled auto…
Xiamen University（厦门大学）；University of Chinese Academy of Sciences（中国科学院大学）；Huawei Cloud & AI（华为云与AI）
Unsupervised domain adaptation (UDA) aims to address the domain-shift problem between a labeled source domain and an unlabeled target domain. Many efforts have been made to address the mismatch between the distributions of training and testing data, but unfortunately, they ignore the task-oriented information across domains and are inflexible to perform well in complicated open-set scenarios. Many efforts have been made to eliminate the mismatch between the distributions of training and testing data by learning domain-invariant representations. However, the learned representations are usually …
Unsupervised Robust Domain Adaptation: Paradigm, Theory and Algorithm
Chongqing Key Laboratory of Bio-perception and Multimodal Intelligent Information Processing（重庆生物感知与多模态智能信息处理重点实验室）；Chongqing University（重庆大学）；School of Microelectronics and Communication Engineering（微电子与通信工程学院）；University of Electronic Science and Technology of China（电子科学与技术大学）；Chongqing Key Laboratory of Image Cognition（重庆图像认知重点实验室）；Chongqing University of Posts and Telecommunications（重庆邮电大学）；School of Data Science（数据科学学院）；Lingnan University Hong Kong（香港岭南大学）；School of Science and Engineering（科学与工程学院）；Chinese University of Hong Kong (Shenzhen)（香港中文大学（深圳））
School of Software, Tsinghua University, China（清华大学软件学院，中国）；Tsinghua University, China（清华大学，中国）；University of California, Berkeley, Berkeley, USA（美国加州大学伯克利分校，伯克利）
Adversarial learning has been embedded into deep networks to learn disentangled and transferable representations for domain adaptation. Existing adversarial domain adaptation methods may not effectively align different domains of multimodal distributions native in classification problems. In this paper, we present conditional adversarial domain adaptation, a principled framework that conditions the adversarial adaptation models on discriminative information conveyed in the classifier predictions. Conditional domain adversarial networks (CDANs) are designed with two novel conditioning strategie…
Many methods have been proposed to solve the domain adaptation problem recently. However, the success of them implicitly funds on the assumption that the information of domains are fully transferrable. If the assumption is not satisfied, the effect of negative transfer may degrade domain adaptation. In this paper, a better learning network has been proposed by considering three tasks - domain adaptation, disentangled representation, and style transfer simultaneously. Firstly, the learned features are disentangled into common parts and specific parts. The common parts represent the transferrabl…
Shandong University（山东大学）；Tel Aviv University（特拉维夫大学）；The Hebrew University of Jerusalem（耶路撒冷希伯来大学）
Unsupervised domain adaptation aims at learning a shared model for two related, but not identical, domains by leveraging supervision from a source domain to an unsupervised target domain. A number of effective domain adaptation approaches rely on the ability to extract discriminative, yet domain-invariant, latent factors which are common to both domains. Extracting latent commonality is also useful for disentanglement analysis, enabling separation between the common and the domain-specific features of both domains. In this paper, we present a method for boosting domain adaptation performance b…
Guangdong University of Technology（广东工业大学）；National University of Singapore（新加坡国立大学）；Tencent Technology (SZ) Co., Ltd.（腾讯科技（深圳）有限公司）；Carnegie Mellon University（卡内基梅隆大学）
Recent years have witnessed tremendous interest in deep learning on graph-structured data. Due to the high cost of collecting labeled graph-structured data, domain adaptation is important to supervised graph learning tasks with limited samples. However, current graph domain adaptation methods are generally adopted from traditional domain adaptation tasks, and the properties of graph-structured data are not well utilized. For example, the observed social networks on different platforms are controlled not only by the different crowd or communities but also by the domain-specific policies and the…
Tsinghua University（清华大学）；Renmin University of China（中国人民大学）
Recent advance in Dense Retrieval (DR) techniques has significantly improved the effectiveness of first-stage retrieval. Trained with large-scale supervised data, DR models can encode queries and documents into a low-dimensional dense space and conduct effective semantic matching. However, previous studies have shown that the effectiveness of DR models would drop by a large margin when the trained DR models are adopted in a target domain that is different from the domain of the labeled data. One of the possible reasons is that the DR model has never seen the target corpus and thus might be inc…
Guangdong University of Technology（广东工业大学）；Foshan University（佛山科学技术学院）
Named entity recognition (NER) for identifying proper nouns in unstructured text is one of the most important and fundamental tasks in natural language processing. However, despite the widespread use of NER models, they still require a large-scale labeled data set, which incurs a heavy burden due to manual annotation. Domain adaptation is one of the most promising solutions to this problem, where rich labeled data from the relevant source domain are utilized to strengthen the generalizability of a model based on the target domain. However, the mainstream cross-domain NER models are still affec…
A2V: A Semi-Supervised Domain Adaptation Framework for Brain Vessel Segmentation via Two-Phase Training Angiography-to-Venography Translation
EURECOM（欧洲通信学院）；University of Siena（锡耶纳大学）；Centre Hospitalier Universitaire de Nice（尼斯大学医院中心）；Centre for Medical Image Computing University College London（伦敦大学学院医学影像计算中心）；e-health Center Universitat Oberta de Catalunya（加泰罗尼亚开放大学电子健康中心）；Sorbonne Université（索邦大学）；CNRS（法国国家科学研究中心）；Inria（法国国家信息与自动化研究所）；Inserm（法国国家健康与医学研究院）；AP-HP（巴黎公共医疗救助机构）
School of Computing（计算学院）；University of Utah（犹他大学）
We describe an approach to domain adaptation that is appropriate exactly in the case when one has enough ``target'' data to do slightly better than just using only ``source'' data. Our approach is incredibly simple, easy to implement as a preprocessing step (10 lines of Perl!) and outperforms state-of-the-art approaches on a range of datasets. Moreover, it is trivially extended to a multi-domain adaptation problem, where one has data from a variety of different domains.
Mythic Inc.（Mythic公司）；Freeus, LLC.（Freeus有限责任公司）；University of Rochester（罗切斯特大学）
Deep neural networks can form high-level hierarchical representations of input data. Various researchers have demonstrated that these representations can be used to enable a variety of useful applications. However, such representations are typically based on the statistics within the data, and may not conform with the semantic representation that may be necessitated by the application. Conditional models are typically used to overcome this challenge, but they require large annotated datasets which are difficult to come by and costly to create. In this paper, we show that semantically-aligned r…
Current domain adaptation methods for face anti-spoofing leverage labeled source domain data and unlabeled target domain data to obtain a promising generalizable decision boundary. However, it is usually difficult for these methods to achieve a perfect domain-invariant liveness feature disentanglement, which may degrade the final classification performance by domain differences in illumination, face category, spoof type, etc. In this work, we tackle cross-scenario face anti-spoofing by proposing a novel domain adaptation method called cyclically disentangled feature translation network (CDFTN)…
University of California, Berkeley（加州大学伯克利分校）；Didi Chuxing（滴滴出行）
In many practical applications, it is often difficult and expensive to obtain enough large-scale labeled data to train deep neural networks to their full capability. Therefore, transferring the learned knowledge from a separate, labeled source domain to an unlabeled or sparsely labeled target domain becomes an appealing alternative. However, direct transfer often results in significant performance decay due to domain shift. Domain adaptation (DA) addresses this problem by minimizing the impact of domain shift between the source and target domains. Multi-source domain adaptation (MDA) is a powe…
Dataset bias remains a significant barrier towards solving real world computer vision tasks. Though deep convolutional networks have proven to be a competitive approach for image classification, a question remains: have these models have solved the dataset bias problem? In general, training or fine-tuning a state-of-the-art deep model on a new domain requires a significant amount of data, which for many applications is simply not available. Transfer of models directly to new domains without adaptation has historically led to poor recognition performance. In this paper, we pose the following qu…
Harbin Insititute of Technology（哈尔滨工业大学）；Macquaire University（麦考瑞大学）
Existing Deep Neural Nets on crops growth prediction mostly rely on availability of a large amount of data. In practice, it is difficult to collect enough high-quality data to utilize the full potential of these deep learning models. In this paper, we construct an innovative network architecture based on domain adaptation learning to predict crops growth curves with limited available crop data. This network architecture overcomes the challenge of data availability by incorporating generated data from the developed crops simulation model. We are the first to use the temporal convolution filters…
Data-efficient domain adaptation with only a few labelled data is desired for many robotic applications, e.g., in grasping detection, the inference skill learned from a grasping dataset is not universal enough to directly apply on various other daily/industrial applications. This paper presents an approach enabling the easy domain adaptation through a novel grasping detection network with confidence-driven semi-supervised learning, where these two components deeply interact with each other. The proposed grasping detection network specially provides a prediction uncertainty estimation mechanism…
Recent neural Text-to-Speech (TTS) models have been shown to perform very well when enough data is available. However, fine-tuning them for new speakers or languages is not straightforward in a low-resource setup. In this paper, we show that by applying minor modifications to a Tacotron model, one can transfer an existing TTS model for new speakers from the same or a different language using only 20 minutes of data. For this purpose, we first introduce a base multi-lingual Tacotron with language-agnostic input, then demonstrate how transfer learning is done for different scenarios of speaker a…
School of Engineering, University of Edinburgh（爱丁堡大学工程学院）；Canon Medical Research Europe Ltd.（佳能医疗欧洲研究有限公司）；The Alan Turing Institute（艾伦·图灵研究所）
Robust cardiac image segmentation is still an open challenge due to the inability of the existing methods to achieve satisfactory performance on unseen data of different domains. Since the acquisition and annotation of medical data are costly and time-consuming, recent work focuses on domain adaptation and generalization to bridge the gap between data from different populations and scanners. In this paper, we propose two data augmentation methods that focus on improving the domain adaptation and generalization abilities of state-to-the-art cardiac segmentation models. In particular, our "Resol…
Technische Universität Berlin（柏林工业大学）；SAP SE（SAP公司）
Face images are subject to many different factors of variation, especially in unconstrained in-the-wild scenarios. For most tasks involving such images, e.g. expression recognition from video streams, having enough labeled data is prohibitively expensive. One common strategy to tackle such a problem is to learn disentangled representations for the different factors of variation of the observed data using adversarial learning. In this paper, we use a formulation of the adversarial loss to learn disentangled representations for face images. The used model facilitates learning on single-task data…
Department of Biomedical Engineering, Yale University（耶鲁大学生物医学工程系）；Department of Radiology & Biomedical Imaging, Yale School of Medicine（耶鲁大学医学院放射科与生物医学成像系）；Department of Electrical Engineering, Yale University（耶鲁大学电气工程系）；Department of Statistics & Data Science, Yale University（耶鲁大学统计与数据科学系）；Yale School of Medicine（耶鲁大学医学院）；Yale University（耶鲁大学）
A deep learning model trained on some labeled data from a certain source domain generally performs poorly on data from different target domains due to domain shifts. Unsupervised domain adaptation methods address this problem by alleviating the domain shift between the labeled source data and the unlabeled target data. In this work, we achieve cross-modality domain adaptation, i.e. between CT and MRI images, via disentangled representations. Compared to learning a one-to-one mapping as the state-of-art CycleGAN, our model recovers a many-to-many mapping between domains to capture the complex c…
College of Information and Computer Sciences（信息与计算机科学学院）；University of Massachusetts（马萨诸塞大学）
Domain adaptation is an essential task in transfer learning to leverage data in one domain to bolster learning in another domain. In this paper, we present a new semi-supervised manifold alignment technique based on a two-step approach of projecting and filtering the source and target domains to low dimensional spaces followed by joining the two spaces. Our proposed approach, filtered manifold alignment (FMA), reduces the computational complexity of previous manifold alignment techniques, is flexible enough to align domains with completely disparate sets of feature and demonstrates state-of-th…
IBM Research（IBM研究院）；MINES ParisTech - PSL Research University（巴黎高等矿业学院 - PSL研究大学）
This paper introduces an ensemble of discriminators that improves the accuracy of a domain adaptation technique for the localization of multiple sound sources. Recently, deep neural networks have led to promising results for this task, yet they require a large amount of labeled data for training. Recording and labeling such datasets is very costly, especially because data needs to be diverse enough to cover different acoustic conditions. In this paper, we leverage acoustic simulators to inexpensively generate labeled training samples. However, models trained on synthetic data tend to perform p…
We focus on the problem of domain adaptation when the goal is shifting the model towards the target distribution, rather than learning domain invariant representations. It has been shown that under the following two assumptions: (a) access to samples from intermediate distributions, and (b) samples being annotated with the amount of change from the source distribution, self-training can be successfully applied on gradually shifted samples to adapt the model toward the target distribution. We hypothesize having (a) is enough to enable iterative self-training to slowly adapt the model to the tar…
F^2TTA: Free-Form Test-Time Adaptation on Cross-Domain Medical Image Classification via Image-Level Disentangled Prompt Tuning
School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai, 200240, China（上海交通大学生物医学工程学院）；Institute of Medical Robotics, Shanghai Jiao Tong University, Shanghai, 200240, China（上海交通大学医学机器人研究所）；School of Computer Science and Engineering, Southeast University, Nanjing, 210096, China（东南大学计算机科学与工程学院）；Shanghai Artificial Intelligence Laboratory, Shanghai, 200000, China（上海人工智能实验室）；Department of Systems and Enterprises, Stevens Institute of Technology, Hoboken, NJ, 07030, United States（史蒂文斯理工学院系统与企业部门）
Disentangled Explanations of Neural Network Predictions by Finding Relevant Subspaces
Technische Universität Berlin (TU Berlin)（柏林工业大学）；Max Planck School of Cognition（马克斯·普朗克认知学学院）；Max Planck Institute for Human Cognitive and Brain Sciences（马克斯·普朗克人类认知与脑科学研究所）；Konrad Zuse School of Excellence in Learning and Intelligent Systems (ELIZA)（康拉德·楚泽学习与智能系统卓越学院）；BASF SE（巴斯夫公司）；Korea University（高丽大学）；Max Planck Institute for Informatics（马克斯·普朗克信息学研究所）；BIFOLD—Berlin Institute for the Foundations of Learning and Data（BIFOLD——柏林学习与数据基础研究所）；Freie Universität Berlin (FU Berlin)（柏林自由大学）
University of Electronic Science and Technology（电子科技大学）
Domain adaptation investigates the problem of leveraging knowledge from a well-labeled source domain to an unlabeled target domain, where the two domains are drawn from different data distributions. Because of the distribution shifts, different target samples have distinct degrees of difficulty in adaptation. However, existing domain adaptation approaches overwhelmingly neglect the degrees of difficulty and deploy exactly the same framework for all of the target samples. Generally, a simple or shadow framework is fast but rough. A sophisticated or deep framework, on the contrary, is accurate b…
Domain adaptation aims to generalize a model from a source domain to tackle tasks in a related but different target domain. Traditional domain adaptation algorithms assume that enough labeled data, which are treated as the prior knowledge are available in the source domain. However, these algorithms will be infeasible when only a few labeled data exist in the source domain, and thus the performance decreases significantly. To address this challenge, we propose a Domain-invariant Graph Learning (DGL) approach for domain adaptation with only a few labeled source samples. Firstly, DGL introduces …
Qualcomm Technologies, Inc.（高通技术公司）；Qualcomm AI Research（高通人工智能研究院）；KAIST（韩国科学技术院）
This paper proposes a novel batch normalization strategy for test-time adaptation. Recent test-time adaptation methods heavily rely on the modified batch normalization, i.e., transductive batch normalization (TBN), which calculates the mean and the variance from the current test batch rather than using the running mean and variance obtained from the source data, i.e., conventional batch normalization (CBN). Adopting TBN that employs test batch statistics mitigates the performance degradation caused by the domain shift. However, re-estimating normalization statistics using test data depends on …
Korea Advanced Institute of Science and Technology (KAIST)（韩国科学技术院（KAIST））
Despite its importance, unsupervised domain adaptation (UDA) on LiDAR semantic segmentation is a task that has not received much attention from the research community. Only recently, a completion-based 3D method has been proposed to tackle the problem and formally set up the adaptive scenarios. However, the proposed pipeline is complex, voxel-based and requires multi-stage inference, which inhibits it for real-time inference. We propose a range image-based, effective and efficient method for solving UDA on LiDAR segmentation. The method exploits class prototypes from the source domain to pseud…
