Conventional domain adaptation (DA) techniques aim to improve domain transferability by learning domain-invariant representations; while concurrently preserving the task-discriminability knowledge gathered from the labeled source data. However, the requirement of simultaneous access to labeled source and unlabeled target renders them unsuitable for the challenging source-free DA setting. The trivial solution of realizing an effective original to generic domain mapping improves transferability but degrades task discriminability. Upon analyzing the hurdles from both theoretical and empirical standpoints, we derive novel insights to show that a mixup between original and corresponding translated generic samples enhances the discriminability-transferability trade-off while duly respecting the privacy-oriented source-free setting. A simple but effective realization of the proposed insights on top of the existing source-free DA approaches yields state-of-the-art performance with faster convergence. Beyond single-source, we also outperform multi-source prior-arts across both classification and semantic segmentation benchmarks.
On the Transferability and Discriminability of Repersentation Learning in Unsupervised Domain Adaptation
National Key Laboratory of Space Integrated Information System, Institute of Software Chinese Academy of Sciences（空间信息集成信息系统国家重点实验室，软件研究所）；University of Chinese Academy of Sciences（中国科学院大学）；National Key Laboratory of Space Integrated Information System, Department of Computer Science and Technology, Tsinghua University（空间信息集成信息系统国家重点实验室，清华大学计算机科学与技术系）；Artificial Intelligence Thrust, Information Hub, Department of Computer Science & Engineering, School of Engineering, Hong Kong University of Science and Technology（人工智能 thrust，信息枢纽，工程学院计算机科学与工程系，香港科学与技术大学）
Geometric Understanding of Discriminability and Transferability for Visual Domain Adaptation
School of Mathematics, Sun Yat-Sen University（中山大学数学学院）；School of Statistics and Mathematics, Guangdong University of Finance and Economics（广东财经大学统计与数学学院）；School of Computer Science, Nanjing University of Posts and Telecommunications（南京邮电大学计算机学院）
Unsupervised Domain Adaptation (UDA) aims to transfer the knowledge from the labeled source domain to the unlabeled target domain in the presence of dataset shift. Most existing methods cannot address the domain alignment and class discrimination well, which may distort the intrinsic data structure for downstream tasks (e.g., classification). To this end, we propose a novel geometry-aware model to learn the transferability and discriminability simultaneously via nuclear norm optimization. We introduce the domain coherence and class orthogonality for UDA from the perspective of subspace geometr…
Huazhong University of Science and Technology（华中科技大学）
Maximum mean discrepancy (MMD) has been widely adopted in domain adaptation to measure the discrepancy between the source and target domain distributions. Many existing domain adaptation approaches are based on the joint MMD, which is computed as the (weighted) sum of the marginal distribution discrepancy and the conditional distribution discrepancy; however, a more natural metric may be their joint probability distribution discrepancy. Additionally, most metrics only aim to increase the transferability between domains, but ignores the discriminability between different classes, which may resu…
School of Informatics, Xiamen University（厦门大学信息学院）；Department of Computer Science and Engineering, The Chinese University of Hong Kong（香港中文大学计算机科学与工程学系）
Recent advances in adaptive object detection have achieved compelling results in virtue of adversarial feature adaptation to mitigate the distributional shifts along the detection pipeline. Whilst adversarial adaptation significantly enhances the transferability of feature representations, the feature discriminability of object detectors remains less investigated. Moreover, transferability and discriminability may come at a contradiction in adversarial adaptation given the complex combinations of objects and the differentiated scene layouts between domains. In this paper, we propose a Hierarch…
Qiushi Academy for Advanced Studies, Zhejiang University（浙江大学求是高等研究院）；College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；Harvard Medical School（哈佛医学院）；Cooperative Medianet Innovation Center, Shanghai Jiao Tong University（上海交通大学媒体网络协同创新中心）；Shanghai Jiao Tong University（上海交通大学）；Shanghai AI Laboratory（上海人工智能实验室）；Collaborative Innovation Center for Artificial Intelligence by MOE and Zhejiang Provincial Government (ZJU)（教育部与浙江省政府人工智能协同创新中心（浙江大学））；Zhejiang Lab（之江实验室）
Domain adaptation methods reduce domain shift typically by learning domain-invariant features. Most existing methods are built on distribution matching, e.g., adversarial domain adaptation, which tends to corrupt feature discriminability. In this paper, we propose Discriminative Radial Domain Adaptation (DRDA) which bridges source and target domains via a shared radial structure. It's motivated by the observation that as the model is trained to be progressively discriminative, features of different categories expand outwards in different directions, forming a radial structure. We show that tra…
Zhejiang University（浙江大学）；Laval University（拉瓦尔大学）；State University of New York at Buffalo（纽约州立大学布法罗分校）
Domain adaptation (DA) aims to transfer discriminative features learned from source domain to target domain. Most of DA methods focus on enhancing feature transferability through domain-invariance learning. However, source-learned discriminability itself might be tailored to be biased and unsafely transferable by spurious correlations, \emph{i.e.}, part of source-specific features are correlated with category labels. We find that standard domain-invariance learning suffers from such correlations and incorrectly transfers the source-specifics. To address this issue, we intervene in the learning…
SPA: A Graph Spectral Alignment Perspective for Domain Adaptation
CUHK-SenseTime Joint Lab, The Chinese University of Hong Kong（香港中文大学）；School of Computer Science and Engineering, Nanyang Technological University（南洋理工大学）；Alibaba Group（阿里巴巴集团）；College of Computer Science and Technology, Zhejiang University（浙江大学）；School of Software Technology, Zhejiang University（浙江大学）；Key Lab of Intelligent Computing based Big Data of Zhejiang Province, Zhejiang University（浙江大学）
MEGVII Technology（旷视科技）；Waseda University（早稻田大学）；University of Edinburgh（爱丁堡大学）；Wenzhou University（温州大学）；ByteDance（字节跳动）
This work simultaneously considers the discriminability and transferability properties of deep representations in the typical supervised learning task, i.e., image classification. By a comprehensive temporal analysis, we observe a trade-off between these two properties. The discriminability keeps increasing with the training progressing while the transferability intensely diminishes in the later training period. From the perspective of information-bottleneck theory, we reveal that the incompatibility between discriminability and transferability is attributed to the over-compression of input in…
Carleton University（卡尔顿大学）；University of Ottawa（渥太华大学）
Adversarial domain adaptation has made impressive advances in transferring knowledge from the source domain to the target domain by aligning feature distributions of both domains. These methods focus on minimizing domain divergence and regard the adaptability, which is measured as the expected error of the ideal joint hypothesis on these two domains, as a small constant. However, these approaches still face two issues: (1) Adversarial domain alignment distorts the original feature distributions, deteriorating the adaptability; (2) Transforming feature representations to be domain-invariant nee…
CVKD-UDA: Cross-View Knowledge Distillation for 3D Unsupervised Domain Adaptive Segmentation
School of Artificial Intelligence, Nanyang Normal University（南阳师范学院人工智能学院）；School of Computer Science and Engineering, Northeastern University（东北大学计算机科学与工程学院）；Fujian Key Laboratory of Sensing and Computing for Smart Cities, School of Informatics, Xiamen University（厦门大学信息学院福建省智慧城市感知与计算重点实验室）
SPA++: Generalized Graph Spectral Alignment for Versatile Domain Adaptation
College of Computer Science（计算机科学学院）；Technology, Zhejiang University（技术学院，浙江大学）；School of Software Technology（软件技术学校）；Electronic Engineering, Hunan University（电子工程，湖南大学）
Cross-domain object detection is more challenging than object classification since multiple objects exist in an image and the location of each object is unknown in the unlabeled target domain. As a result, when we adapt features of different objects to enhance the transferability of the detector, the features of the foreground and the background are easy to be confused, which may hurt the discriminability of the detector. Besides, previous methods focused on category adaptation but ignored another important part for object detection, i.e., the adaptation on bounding box regression. To this end…
National University of Defense Technology（国防科技大学）；Harbin Institute of Technology（哈尔滨工业大学）；Dalian University of Technology（大连理工大学）
Nuclear norm maximization has shown the power to enhance the transferability of unsupervised domain adaptation model (UDA) in an empirical scheme. In this paper, we identify a new property termed equity, which indicates the balance degree of predicted classes, to demystify the efficacy of nuclear norm maximization for UDA theoretically. With this in mind, we offer a new discriminability-and-equity maximization paradigm built on squares loss, such that predictions are equalized explicitly. To verify its feasibility and flexibility, two new losses termed Class Weighted Squares Maximization (CWSM…
Dalian University of Technology（大连理工大学）；Indiana University-Purdue University Indianapolis（印第安纳大学-普渡大学印第安纳波利斯分校）
Existing domain adaptation methods aim to reduce the distributional difference between the source and target domains and respect their specific discriminative information, by establishing the Maximum Mean Discrepancy (MMD) and the discriminative distances. However, they usually accumulate to consider those statistics and deal with their relationships by estimating parameters blindly. This paper theoretically proves two essential facts: 1) minimizing the MMD equals to maximize the source and target intra-class distances respectively but jointly minimize their variance with some implicit weights…
School of Software, Tsinghua University, China（清华大学软件学院，中国）；Tsinghua University, China（清华大学，中国）；University of California, Berkeley, Berkeley, USA（美国加州大学伯克利分校，伯克利）
Adversarial learning has been embedded into deep networks to learn disentangled and transferable representations for domain adaptation. Existing adversarial domain adaptation methods may not effectively align different domains of multimodal distributions native in classification problems. In this paper, we present conditional adversarial domain adaptation, a principled framework that conditions the adversarial adaptation models on discriminative information conveyed in the classifier predictions. Conditional domain adversarial networks (CDANs) are designed with two novel conditioning strategie…
Sun Yat-Sen University（中山大学）；JiaYing University（嘉应学院）；Guangzhou University（广州大学）
Unsupervised domain adaptation is effective in leveraging the rich information from the source domain to the unsupervised target domain. Though deep learning and adversarial strategy make an important breakthrough in the adaptability of features, there are two issues to be further explored. First, the hard-assigned pseudo labels on the target domain are risky to the intrinsic data structure. Second, the batch-wise training manner in deep learning limits the description of the global structure. In this paper, a Riemannian manifold learning framework is proposed to achieve transferability and di…
This work introduces the novel task of Source-free Multi-target Domain Adaptation and proposes adaptation framework comprising of \textbf{Co}nsistency with \textbf{N}uclear-Norm Maximization and \textbf{Mix}Up knowledge distillation (\textit{CoNMix}) as a solution to this problem. The main motive of this work is to solve for Single and Multi target Domain Adaptation (SMTDA) for the source-free paradigm, which enforces a constraint where the labeled source data is not available during target adaptation due to various privacy-related restrictions on data sharing. The source-free approach leverag…
Sun Yat-Sen University（中山大学）；City University of Hong Kong（香港城市大学）
Unsupervised domain adaptation is effective in leveraging rich information from a labeled source domain to an unlabeled target domain. Though deep learning and adversarial strategy made a significant breakthrough in the adaptability of features, there are two issues to be further studied. First, hard-assigned pseudo labels on the target domain are arbitrary and error-prone, and direct application of them may destroy the intrinsic data structure. Second, batch-wise training of deep learning limits the characterization of the global structure. In this paper, a Riemannian manifold learning framew…
University of Chinese Academy of Sciences（中国科学院大学）
We present Domain Contrast (DC), a simple yet effective approach inspired by contrastive learning for training domain adaptive detectors. DC is deduced from the error bound minimization perspective of a transferred model, and is implemented with cross-domain contrast loss which is plug-and-play. By minimizing cross-domain contrast loss, DC guarantees the transferability of detectors while naturally alleviating the class imbalance issue in the target domain. DC can be applied at either image level or region level, consistently improving detectors' transferability and discriminability. Extensive…
Although achieving remarkable progress, it is very difficult to induce a supervised classifier without any labeled data. Unsupervised domain adaptation is able to overcome this challenge by transferring knowledge from a labeled source domain to an unlabeled target domain. Transferability and discriminability are two key criteria for characterizing the superiority of feature representations to enable successful domain adaptation. In this paper, a novel method called \textit{learning TransFerable and Discriminative Features for unsupervised domain adaptation} (TFDF) is proposed to optimize these…
Noise May Contain Transferable Knowledge: Understanding Semi-supervised Heterogeneous Domain Adaptation from an Empirical Perspective
Beijing Teleinfo Technology Company Ltd.（北京 Teleinfo 科技有限公司）；China Academy of Information and Communications Technology（中国信息通信研究院）；Inspur Computer Technology Co., Ltd.（浪潮计算机技术有限公司）；Southern University of Science and Technology（南方科技大学）；Research Institute of Industrial Internet of Things（工业互联网物联网研究院）；Hong Kong University of Science and Technology（香港科技大学）；WeBank（微众银行）
Enhancing Adversarial Transferability by Balancing Exploration and Exploitation with Gradient-Guided Sampling
School of Computer Science & Software Engineering, Shenzhen University, China（深圳大学计算机科学与软件工程学院）；Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ), Shenzhen, China（广东省人工智能与数字经济发展实验室（深圳））；Guangdong Provincial Key Laboratory of Intelligent Information Processing, Shenzhen University, China（广东省智能信息处理省级重点实验室）；School of Computer Science, University of Exeter, U.K.（埃克塞特大学计算机科学学院）；Department of Computing and Information Technology, Great Bay University, China（大鹏大学计算与信息技术系）；Computer Vision Institute, School of Artificial Intelligence, Shenzhen University, China（人工智能学院计算机视觉研究所）
Yonsei University（延世大学）；Electronics and Telecommunications Research Institute (ETRI)（韩国电子通信研究院）
Domain adaptation for object detection (DAOD) has recently drawn much attention owing to its capability of detecting target objects without any annotations. To tackle the problem, previous works focus on aligning features extracted from partial levels (e.g., image-level, instance-level, RPN-level) in a two-stage detector via adversarial training. However, individual levels in the object detection pipeline are closely related to each other and this inter-level relation is unconsidered yet. To this end, we introduce a novel framework for DAOD with three proposed components: Multi-scale-aware Unc…
Without Paired Labeled Data: End-to-End Self-Supervised Learning for Drone-view Geo-Localization
State Key Laboratory for Strength and Vibration of Mechanical Structures（强度与振动机械结构国家重点实验室）；Shaanxi Key Laboratory of Environment and Control for Flight Vehicle（陕西省飞行器环境与控制重点实验室）；School of Aerospace Engineering（航空航天工程学院）；Xi’an Jiaotong University（西安交通大学）；Institute of Automation（自动化研究所）；Chinese Academy of Sciences（中国科学院）；School of Artificial Intelligence（人工智能学院）；University of Chinese Academy of Sciences（中国科学院大学）；Peng Cheng Laboratory（鹏城实验室）
Most existing domain adaptive object detection methods exploit adversarial feature alignment to adapt the model to a new domain. Recent advances in adversarial feature alignment strives to reduce the negative effect of alignment, or negative transfer, that occurs because the distribution of features varies depending on the category of objects. However, by analyzing the features of the anchor-free one-stage detector, in this paper, we find that negative transfer may occur because the feature distribution varies depending on the regression value for the offset to the bounding box as well as the …
City University of Hong Kong（香港城市大学）；Kingsoft Cloud（金山云）
In this paper, we quest the capability of transferring the quality of natural scene images to the images that are not acquired by optical cameras (e.g., screen content images, SCIs), rooted in the widely accepted view that the human visual system has adapted and evolved through the perception of natural environment. Here, we develop the first unsupervised domain adaptation based no reference quality assessment method for SCIs, leveraging rich subjective ratings of the natural images (NIs). In general, it is a non-trivial task to directly transfer the quality prediction model from NIs to a new …
