This work addresses the unsupervised domain adaptation problem, especially in the case of class labels in the target domain being only a subset of those in the source domain. Such a partial transfer setting is realistic but challenging and existing methods always suffer from two key problems, negative transfer and uncertainty propagation. In this paper, we build on domain adversarial learning and propose a novel domain adaptation method BA$^3$US with two new techniques termed Balanced Adversarial Alignment (BAA) and Adaptive Uncertainty Suppression (AUS), respectively. On one hand, negative transfer results in misclassification of target samples to the classes only present in the source domain. To address this issue, BAA pursues the balance between label distributions across domains in a fairly simple manner. Specifically, it randomly leverages a few source samples to augment the smaller target domain during domain alignment so that classes in different domains are symmetric. On the other hand, a source sample would be denoted as uncertain if there is an incorrect class that has a relatively high prediction score, and such uncertainty easily propagates to unlabeled target data around it during alignment, which severely deteriorates adaptation performance. Thus we present AUS that emphasizes uncertain samples and exploits an adaptive weighted complement entropy objective to encourage incorrect classes to have uniform and low prediction scores. Experimental results on multiple benchmarks demonstrate our BA$^3$US surpasses state-of-the-arts for partial domain adaptation tasks. Code is available at \url{https://github.com/tim-learn/BA3US}.
pFedBBN: A Personalized Federated Test-Time Adaptation with Balanced Batch Normalization for Class-Imbalanced Data
Center for Computational & Data Sciences (CCDS)（计算与数据科学中心）；Independent University（独立大学）；University of Central Florida（中央佛罗里达大学）；Curtin University（Curtin大学）；Umeå University（乌梅拉大学）
ARC Centre of Excellence for Robotic Vision（澳大利亚研究委员会机器人视觉卓越中心）；Monash University（莫纳什大学）
Domain adaptation and generative modelling have collectively mitigated the expensive nature of data collection and labelling by leveraging the rich abundance of accurate, labelled data in simulation environments. In this work, we study the performance gap that exists between representations optimised for localisation on simulation environments and the application of such representations in a real-world setting. Our method exploits the shared geometric similarities between simulation and real-world environments whilst maintaining invariance towards visual discrepancies. This is achieved by opti…
University of Central Florida（中佛罗里达大学）；University of California at Merced（加州大学默塞德分校）；Google（谷歌）
Object frequency in the real world often follows a power law, leading to a mismatch between datasets with long-tailed class distributions seen by a machine learning model and our expectation of the model to perform well on all classes. We analyze this mismatch from a domain adaptation point of view. First of all, we connect existing class-balanced methods for long-tailed classification to target shift, a well-studied scenario in domain adaptation. The connection reveals that these methods implicitly assume that the training data and test data share the same class-conditioned distribution, whic…
Towards Real-World Test-Time Adaptation: Tri-Net Self-Training with Balanced Normalization
South China University of Technology（华南理工大学）；Institute for Infocomm Research, A*STAR（新加坡科技研究局信息通信研究所）；School of Data Science, The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳）数据科学学院）
In this paper, we introduce a collaborative training algorithm of balanced random forests with convolutional neural networks for domain adaptation tasks. In real scenarios, most domain adaptation algorithms face the challenges from noisy, insufficient training data and open set categorization. In such cases, conventional methods suffer from overfitting and fail to successfully transfer the knowledge of the source to the target domain. To address these issues, the following two techniques are proposed. First, we introduce the optimized decision tree construction method with convolutional neural…
Carnegie Mellon University（卡内基梅隆大学）；NVIDIA（英伟达）；General Motors R & D（通用汽车研发中心）
Recent deep networks achieved state of the art performance on a variety of semantic segmentation tasks. Despite such progress, these models often face challenges in real world `wild tasks' where large difference between labeled training/source data and unseen test/target data exists. In particular, such difference is often referred to as `domain gap', and could cause significantly decreased performance which cannot be easily remedied by further increasing the representation power. Unsupervised domain adaptation (UDA) seeks to overcome such problem without target domain labels. In this paper, w…
Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Institute of High Performance Computing, A*STAR（新加坡科技研究局高性能计算研究所）；Nanyang Technological University（南洋理工大学）
Transfer learning has achieved promising results by leveraging knowledge from the source domain to annotate the target domain which has few or none labels. Existing methods often seek to minimize the distribution divergence between domains, such as the marginal distribution, the conditional distribution or both. However, these two distances are often treated equally in existing algorithms, which will result in poor performance in real applications. Moreover, existing methods usually assume that the dataset is balanced, which also limits their performances on imbalanced tasks that are quite com…
Adaptive and Balanced Re-initialization for Long-timescale Continual Test-time Domain Adaptation
Hong Kong Polytechnic University（香港理工大学）；Eastern Institute for Advanced Study（东方高级研究所）；Ant Group（蚂蚁集团）；Australian National University（澳大利亚国立大学）；CSIRO（澳大利亚科学院）；The University of Hong Kong（香港大学）
Microsoft（微软）；IIT Kanpur（印度理工学院坎普尔分校）；Oracle（甲骨文公司）；University of Bath（巴斯大学）
Understanding unsupervised domain adaptation has been an important task that has been well explored. However, the wide variety of methods have not analyzed the role of a classifier's performance in detail. In this paper, we thoroughly examine the role of a classifier in terms of matching source and target distributions. We specifically investigate the classifier ability by matching a) the distribution of features, b) probabilistic uncertainty for samples and c) certainty activation mappings. Our analysis suggests that using these three distributions does result in a consistently improved perfo…
Tongji University（同济大学）；The University of Sydney（悉尼大学）；University of Science and Technology of China（中国科学技术大学）；University of Science and Technology Beijing（北京科技大学）；JD Explore Academy（京东探索研究院）
Source-free Domain Adaptation (SFDA) aims to adapt a pre-trained source model to the unlabeled target domain without accessing the well-labeled source data, which is a much more practical setting due to the data privacy, security, and transmission issues. To make up for the absence of source data, most existing methods introduced feature prototype based pseudo-labeling strategies to realize self-training model adaptation. However, feature prototypes are obtained by instance-level predictions based feature clustering, which is category-biased and tends to result in noisy labels since the visual…
Unsupervised domain adaptation (UDA) deals with the problem of classifying unlabeled target domain data while labeled data is only available for a different source domain. Unfortunately, commonly used classification methods cannot fulfill this task adequately due to the domain gap between the source and target data. In this paper, we propose a novel uncertainty-aware domain adaptation setup that models uncertainty as a multivariate Gaussian distribution in feature space. We show that our proposed uncertainty measure correlates with other common uncertainty quantifications and relates to smooth…
Georgia Tech（佐治亚理工学院）；Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）；Boston University（波士顿大学）
Generalizing deep neural networks to new target domains is critical to their real-world utility. In practice, it may be feasible to get some target data labeled, but to be cost-effective it is desirable to select a maximally-informative subset via active learning (AL). We study the problem of AL under a domain shift, called Active Domain Adaptation (Active DA). We demonstrate how existing AL approaches based solely on model uncertainty or diversity sampling are less effective for Active DA. We propose Clustering Uncertainty-weighted Embeddings (CLUE), a novel label acquisition strategy for Act…
University of Trento（特伦托大学）；Fondazione Bruno Kessler（布鲁诺·凯塞勒基金会）；Aalto University（阿尔托大学）；NVIDIA（英伟达）
Source-free domain adaptation (SFDA) aims to adapt a classifier to an unlabelled target data set by only using a pre-trained source model. However, the absence of the source data and the domain shift makes the predictions on the target data unreliable. We propose quantifying the uncertainty in the source model predictions and utilizing it to guide the target adaptation. For this, we construct a probabilistic source model by incorporating priors on the network parameters inducing a distribution over the model predictions. Uncertainties are estimated by employing a Laplace approximation and inco…
Samsung SDS Technology Research（三星SDS技术研究院）；Seoul National University of Science and Technology（首尔国立科技大学）
Most unsupervised domain adaptation (UDA) methods assume that labeled source images are available during model adaptation. However, this assumption is often infeasible owing to confidentiality issues or memory constraints on mobile devices. Some recently developed approaches do not require source images during adaptation, but they show limited performance on perturbed images. To address these problems, we propose a novel source-free UDA method that uses only a pre-trained source model and unlabeled target images. Our method captures the aleatoric uncertainty by incorporating data augmentation …
Uncertainty Awareness on Unsupervised Domain Adaptation for Time Series Data
College of Computing and Data Science, Nanyang Technological University (NTU), Singapore（南洋理工大学计算与数据科学学院）；School of Computing and Artificial Intelligence, Jiangxi University of Finance and Economics（江西财经大学计算机与人工智能学院）；Institute for Infocomm Research (I 2 R), A*STAR, Singapore（信息通信研究所（I2R），A*STAR，新加坡）；School of Artificial Intelligence, Nanjing University of Information Science and Technology（南京信息科学技术大学人工智能学院）
Unsupervised domain adaptation seeks to learn an invariant and discriminative representation for an unlabeled target domain by leveraging the information of a labeled source dataset. We propose to improve the discriminative ability of the target domain representation by simultaneously learning tightly clustered target representations while encouraging that each cluster is assigned to a unique and different class from the source. This strategy alleviates the effects of negative transfer when combined with adversarial domain matching between source and target representations. Our approach is rob…
Qiushi Academy for Advanced Studies, Zhejiang University（浙江大学求是高等研究院）；College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；State University of New York at Buffalo（纽约州立大学布法罗分校）；Zhejiang University School of Medicine（浙江大学医学院）
Domain adaptation is an important technique to alleviate performance degradation caused by domain shift, e.g., when training and test data come from different domains. Most existing deep adaptation methods focus on reducing domain shift by matching marginal feature distributions through deep transformations on the input features, due to the unavailability of target domain labels. We show that domain shift may still exist via label distribution shift at the classifier, thus deteriorating model performances. To alleviate this issue, we propose an approximate joint distribution matching scheme by…
Present domain adaptation methods usually perform explicit representation alignment by simultaneously accessing the source data and target data. However, the source data are not always available due to the privacy preserving consideration or bandwidth limitation. Source-free domain adaptation aims to solve the above problem by performing domain adaptation without accessing the source data. The adaptation paradigm is receiving more and more attention in recent years, and multiple works have been proposed for unsupervised source-free domain adaptation. However, without utilizing any supervised s…
Unsupervised domain adaptation (UDA) deals with the adaptation of models from a given source domain with labeled data to an unlabeled target domain. In this paper, we utilize the inherent prediction uncertainty of a model to accomplish the domain adaptation task. The uncertainty is measured by Monte-Carlo dropout and used for our proposed Uncertainty-based Filtering and Feature Alignment (UFAL) that combines an Uncertain Feature Loss (UFL) function and an Uncertainty-Based Filtering (UBF) approach for alignment of features in Euclidean space. Our method surpasses recently proposed architecture…
Unsupervised domain adaptation (UDA) deals with the adaptation process of a model to an unlabeled target domain while annotated data is only available for a given source domain. This poses a challenging task, as the domain shift between source and target instances deteriorates a model's performance when not addressed. In this paper, we propose UBR$^2$S - the Uncertainty-Based Resampling and Reweighting Strategy - to tackle this problem. UBR$^2$S employs a Monte Carlo dropout-based uncertainty estimate to obtain per-class probability distributions, which are then used for dynamic resampling of …
Recent advances in domain adaptation establish that requiring a low risk on the source domain and equal feature marginals degrade the adaptation's performance. At the same time, empirical evidence shows that incorporating an unsupervised target domain term that pushes decision boundaries away from the high-density regions, along with relaxed alignment, improves adaptation. In this paper, we theoretically justify such observations via a new bound on the target risk, and we connect two notions of relaxation for divergence, namely $β-$relaxed divergences and localization. This connection allows u…
Conformal Uncertainty Indicator for Continual Test-Time Adaptation
New Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所新模式识别实验室）；China Nuclear Power Engineering Co., LTD（中国核电工程有限公司）；College of Intelligence and Computing, Tianjin University（天津大学智能与计算学部）；School of Computer Science & Engineering, LinYi University（临沂大学计算机科学与工程学院）；School of Electric & Information Engineering, Suzhou University of Science and Technology（苏州科技大学电子与信息工程学院）
Data-efficient domain adaptation with only a few labelled data is desired for many robotic applications, e.g., in grasping detection, the inference skill learned from a grasping dataset is not universal enough to directly apply on various other daily/industrial applications. This paper presents an approach enabling the easy domain adaptation through a novel grasping detection network with confidence-driven semi-supervised learning, where these two components deeply interact with each other. The proposed grasping detection network specially provides a prediction uncertainty estimation mechanism…
Computer Science Department, University of California, Los Angeles, CA, USA（加州大学洛杉矶分校计算机科学系）；Mechanical and Aerospace Engineering Department, University of California, Los Angeles, CA, USA（加州大学洛杉矶分校机械与航空航天工程系）；Bioengineering Department, University of California, Los Angeles, CA, USA（加州大学洛杉矶分校生物工程系）；Jonsson Comprehensive Cancer Center, University of California, Los Angeles, CA, USA（加州大学洛杉矶分校乔纳森综合癌症中心）
Seoul National University of Science and Technology（首尔国立科技大学）；Samsung SDS Technology Research（三星SDS技术研究院）
Unsupervised domain adaptation (UDA) aims to improve the prediction performance in the target domain under distribution shifts from the source domain. The key principle of UDA is to minimize the divergence between the source and the target domains. To follow this principle, many methods employ a domain discriminator to match the feature distributions. Some recent methods evaluate the discrepancy between two predictions on target samples to detect those that deviate from the source distribution. However, their performance is limited because they either match the marginal distributions or measur…
Janus-LoRA: A Balanced Low-Rank Adaptation for Continual Learning
School of Computer Science and Technology, Tongji University, Shanghai, China（同济大学计算机科学与技术学院，上海，中国）；School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu, China（电子科技大学计算机科学与工程学院，成都，中国）；Shanghai Innovation Institute, Shanghai, China（上海创新研究院，上海，中国）；Independent Researcher（独立研究者）
Texas A&M University（德克萨斯A&M大学）；Brookhaven National Laboratory（布鲁克海文国家实验室）
Low-Rank Adaptation (LoRA) offers a cost-effective solution for fine-tuning large language models (LLMs), but it often produces overconfident predictions in data-scarce few-shot settings. To address this issue, several classical statistical learning approaches have been repurposed for scalable uncertainty-aware LoRA fine-tuning. However, these approaches neglect how input characteristics affect the predictive uncertainty estimates. To address this limitation, we propose Contextual Low-Rank Adaptation (C-LoRA) as a novel uncertainty-aware and parameter efficient fine-tuning approach, by develop…
Prior-free Balanced Replay: Uncertainty-guided Reservoir Sampling for Long-Tailed Continual Learning
Shenzhen Research Institute of Big Data (SRIBD)（深圳大数据研究院）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；The Hong Kong University of Science and Technology（香港科技大学）；The Hong Kong Polytechnic University（香港理工大学）
Active domain adaptation (DA) aims to maximally boost the model adaptation on a new target domain by actively selecting limited target data to annotate, whereas traditional active learning methods may be less effective since they do not consider the domain shift issue. Despite active DA methods address this by further proposing targetness to measure the representativeness of target domain characteristics, their predictive uncertainty is usually based on the prediction of deterministic models, which can easily be miscalibrated on data with distribution shift. Considering this, we propose a \tex…
Standard Unsupervised Domain Adaptation (UDA) methods assume the availability of both source and target data during the adaptation. In this work, we investigate Source-free Unsupervised Domain Adaptation (SF-UDA), a specific case of UDA where a model is adapted to a target domain without access to source data. We propose a novel approach for the SF-UDA setting based on a loss reweighting strategy that brings robustness against the noise that inevitably affects the pseudo-labels. The classification loss is reweighted based on the reliability of the pseudo-labels that is measured by estimating t…
Unsupervised domain adaptive object detection aims to adapt detectors from a labelled source domain to an unlabelled target domain. Most existing works take a two-stage strategy that first generates region proposals and then detects objects of interest, where adversarial learning is widely adopted to mitigate the inter-domain discrepancy in both stages. However, adversarial learning may impair the alignment of well-aligned samples as it merely aligns the global distributions across domains. To address this issue, we design an uncertainty-aware domain adaptation network (UaDAN) that introduces …
Test-time adaptation (TTA) is an emerging paradigm that addresses distributional shifts between training and testing phases without additional data acquisition or labeling cost; only unlabeled test data streams are used for continual model adaptation. Previous TTA schemes assume that the test samples are independent and identically distributed (i.i.d.), even though they are often temporally correlated (non-i.i.d.) in application scenarios, e.g., autonomous driving. We discover that most existing TTA methods fail dramatically under such scenarios. Motivated by this, we present a new test-time a…
Queen Mary University of London（伦敦玛丽女王大学）；Zhejiang Lab（之江实验室）；Shanghai Jiao Tong University（上海交通大学）
Domain adaptation (DA) aims to transfer knowledge learned from a labeled source domain to an unlabeled or a less labeled but related target domain. Ideally, the source and target distributions should be aligned to each other equally to achieve unbiased knowledge transfer. However, due to the significant imbalance between the amount of annotated data in the source and target domains, usually only the target distribution is aligned to the source domain, leading to adapting unnecessary source specific knowledge to the target domain, i.e., biased domain adaptation. To resolve this problem, in this…
UCL（伦敦大学学院）；UCLH NHS Foundation Trust（伦敦大学学院医院NHS基金会信托）
The need for training data can impede the adoption of novel imaging modalities for learning-based medical image analysis. Domain adaptation methods partially mitigate this problem by translating training data from a related source domain to a novel target domain, but typically assume that a one-to-one translation is possible. Our work addresses the challenge of adapting to a more informative target domain where multiple target samples can emerge from a single source sample. In particular we consider translating from mp-MRI to VERDICT, a richer MRI modality involving an optimized acquisition pr…
It is expensive to generate real-life image labels and there is a domain gap between real-life and simulated images, hence a model trained on the latter cannot adapt to the former. Solving this can totally eliminate the need for labeling real-life datasets completely. Class balanced self-training is one of the existing techniques that attempt to reduce the domain gap. Moreover, augmenting RGB with flow maps has improved performance in simple semantic segmentation and geometry is preserved across domains. Hence, by augmenting images with dense optical flow map, domain adaptation in semantic seg…
The performance of automatic speech recognition (ASR) systems typically degrades significantly when the training and test data domains are mismatched. In this paper, we show that self-training (ST) combined with an uncertainty-based pseudo-label filtering approach can be effectively used for domain adaptation. We propose DUST, a dropout-based uncertainty-driven self-training technique which uses agreement between multiple predictions of an ASR system obtained for different dropout settings to measure the model's uncertainty about its prediction. DUST excludes pseudo-labeled data with high unce…
Open-set domain adaptation (OSDA) has gained considerable attention in many visual recognition tasks. However, most existing OSDA approaches are limited due to three main reasons, including: (1) the lack of essential theoretical analysis of generalization bound, (2) the reliance on the coexistence of source and target data during adaptation, and (3) failing to accurately estimate the uncertainty of model predictions. We propose a Progressive Graph Learning (PGL) framework that decomposes the target hypothesis space into the shared and unknown subspaces, and then progressively pseudo-labels the…
Nanjing University of Posts and Telecommunications（南京邮电大学）；Southeast University（东南大学）；Temple University（天普大学）
Entropy minimization has been widely used in unsupervised domain adaptation (UDA). However, existing works reveal that entropy minimization only may result into collapsed trivial solutions. In this paper, we propose to avoid trivial solutions by further introducing diversity maximization. In order to achieve the possible minimum target risk for UDA, we show that diversity maximization should be elaborately balanced with entropy minimization, the degree of which can be finely controlled with the use of deep embedded validation in an unsupervised manner. The proposed minimal-entropy diversity ma…
Uncertainty-Aware Test-Time Adaptation for Cross-Region Spatio-Temporal Fusion of Land Surface Temperature
INSA CVL, Université d’Orléans, PRISME UR 4229（INSA CVL，奥尔良大学，PRISME UR 4229）；Université d’Orléans, INSA CVL, PRISME UR 4229（奥尔良大学，INSA CVL，PRISME UR 4229）；Université d'Orléans, CEDETE, UR 1210（奥尔良大学，CEDETE，UR 1210）
