Semantic segmentation models based on convolutional neural networks have recently displayed remarkable performance for a multitude of applications. However, these models typically do not generalize well when applied on new domains, especially when going from synthetic to real data. In this paper we address the problem of unsupervised domain adaptation (UDA), which attempts to train on labelled data from one domain (source domain), and simultaneously learn from unlabelled data in the domain of interest (target domain). Existing methods have seen success by training on pseudo-labels for these unlabelled images. Multiple techniques have been proposed to mitigate low-quality pseudo-labels arising from the domain shift, with varying degrees of success. We propose DACS: Domain Adaptation via Cross-domain mixed Sampling, which mixes images from the two domains along with the corresponding labels and pseudo-labels. These mixed samples are then trained on, in addition to the labelled data itself. We demonstrate the effectiveness of our solution by achieving state-of-the-art results for GTA5 to Cityscapes, a common synthetic-to-real semantic segmentation benchmark for UDA.
Revisiting Cross-Attention Mechanisms: Leveraging Beneficial Noise for Domain-Adaptive Learning
School of Engineering, Westlake University（西lake大学工程学院）；Centre for Artificial Intelligence and Robotics (CAIR), Hong Kong Institute of Science and Innovation, Chinese Academy of Sciences (HKISI-CAS)（香港科学院人工智能与机器人中心）；State Key Laboratory of Multimodal Artificial Intelligence Systems (MAIS), Institute of Automation, Chinese Academy of Sciences (CASIA)（多模态人工智能系统国家重点实验室）；School of Artificial Intelligence, University of Chinese Academy of Sciences (UCAS)（中国科学院大学人工智能学院）；Alibaba Group（阿里巴巴集团）
We propose a novel domain adaptive action detection approach and a new adaptation protocol that leverages the recent advancements in image-level unsupervised domain adaptation (UDA) techniques and handle vagaries of instance-level video data. Self-training combined with cross-domain mixed sampling has shown remarkable performance gain in semantic segmentation in UDA (unsupervised domain adaptation) context. Motivated by this fact, we propose an approach for human action detection in videos that transfers knowledge from the source domain (annotated dataset) to the target domain (unannotated dat…
Georgia Institute of Technology（佐治亚理工学院）；Baidu USA（百度美国）
The main progress for action segmentation comes from densely-annotated data for fully-supervised learning. Since manual annotation for frame-level actions is time-consuming and challenging, we propose to exploit auxiliary unlabeled videos, which are much easier to obtain, by shaping this problem as a domain adaptation (DA) problem. Although various DA techniques have been proposed in recent years, most of them have been developed only for the spatial direction. Therefore, we propose Mixed Temporal Domain Adaptation (MTDA) to jointly align frame- and video-level embedded feature spaces across d…
MRC London Institute of Medical Sciences（MRC伦敦医学科学研究所）；Imperial College London（帝国理工学院）；National Heart Research Institute Singapore（新加坡国家心脏研究所）；National Heart Center Singapore（新加坡国家心脏中心）；Technical University of Munich（慕尼黑工业大学）
Optimising the analysis of cardiac structure and function requires accurate 3D representations of shape and motion. However, techniques such as cardiac magnetic resonance imaging are conventionally limited to acquiring contiguous cross-sectional slices with low through-plane resolution and potential inter-slice spatial misalignment. Super-resolution in medical imaging aims to increase the resolution of images but is conventionally trained on features from low resolution datasets and does not super-resolve corresponding segmentations. Here we propose a semi-supervised multi-task generative adve…
Dalian University of Technology（大连理工大学）；Data Storage and Intelligent Vision Technical Research Dept Huawei Cloud（华为云数据存储与智能视觉技术研究部）；Key Lab of Machine Perception Peking University（北京大学机器感知重点实验室）
Data-driven based approaches, in spite of great success in many tasks, have poor generalization when applied to unseen image domains, and require expensive cost of annotation especially for dense pixel prediction tasks such as semantic segmentation. Recently, both unsupervised domain adaptation (UDA) from large amounts of synthetic data and semi-supervised learning (SSL) with small set of labeled data have been studied to alleviate this issue. However, there is still a large gap on performance compared to their supervised counterparts. We focus on a more practical setting of semi-supervised do…
Uncertainty Awareness on Unsupervised Domain Adaptation for Time Series Data
College of Computing and Data Science, Nanyang Technological University (NTU), Singapore（南洋理工大学计算与数据科学学院）；School of Computing and Artificial Intelligence, Jiangxi University of Finance and Economics（江西财经大学计算机与人工智能学院）；Institute for Infocomm Research (I 2 R), A*STAR, Singapore（信息通信研究所（I2R），A*STAR，新加坡）；School of Artificial Intelligence, Nanjing University of Information Science and Technology（南京信息科学技术大学人工智能学院）
Graz University of Technology（格拉茨技术大学）；Max-Planck-Institute for Informatics（马克斯·普朗克信息学研究所）；Southeast University（东南大学）；Goethe University Frankfurt（法兰克福大学）
Although action recognition has achieved impressive results over recent years, both collection and annotation of video training data are still time-consuming and cost intensive. Therefore, image-to-video adaptation has been proposed to exploit labeling-free web image source for adapting on unlabeled target videos. This poses two major challenges: (1) spatial domain shift between web images and video frames; (2) modality gap between image and video data. To address these challenges, we propose Cycle Domain Adaptation (CycDA), a cycle-based approach for unsupervised image-to-video domain adaptat…
University of Bristol（布里斯托大学）；Singapore Management University（新加坡管理大学）；Fudan University（复旦大学）；City University of Hong Kong（香港城市大学）
Cross-modal recipe retrieval has attracted research attention in recent years, thanks to the availability of large-scale paired data for training. Nevertheless, obtaining adequate recipe-image pairs covering the majority of cuisines for supervised learning is difficult if not impossible. By transferring knowledge learnt from a data-rich cuisine to a data-scarce cuisine, domain adaptation sheds light on this practical problem. Nevertheless, existing works assume recipes in source and target domains are mostly originated from the same cuisine and written in the same language. This paper studies …
State Key Laboratory of Fire Science, University of Science and Technology of China（中国科学技术大学火灾科学国家重点实验室）
In this paper, a deep domain adaptation based method for video smoke detection is proposed to extract a powerful feature representation of smoke. Due to the smoke image samples limited in scale and diversity for deep CNN training, we systematically produced adequate synthetic smoke images with a wide variation in the smoke shape, background and lighting conditions. Considering that the appearance gap (dataset bias) between synthetic and real smoke images degrades significantly the performance of the trained model on the test set composed fully of real images, we build deep architectures based …
STRADAViT: Self-Supervised Domain Adaptation of Vision Transformer Backbones for Radio Astronomy
organization= Institute of Space Sciences；Astronomy , addressline= University of Malta , city= Msida , postcode= MSD2080 , country= Malta；organization= School of Physics；Astronomy, Royal Observatory , addressline= University of Edinburgh , city= Edinburgh , country= United Kingdom
The Johns Hopkins University Applied Physics Lab（约翰斯·霍普金斯大学应用物理实验室）
In practical scenarios, it is often the case that the available training data within the target domain only exist for a limited number of classes, with the remaining classes only available within surrogate domains. We show that including the target domain in training when there exist disjoint classes between the target and surrogate domains creates significant negative transfer, and causes performance to significantly decrease compared to training without the target domain at all. We hypothesize that this negative transfer is due to an intermediate shortcut that only occurs when multiple sourc…
The Chinese University of Hong Kong（香港中文大学）；UC Berkeley（加州大学伯克利分校）；ICSI（国际计算机科学研究所）；Google Inc.（谷歌公司）
A typical domain adaptation approach is to adapt models trained on the annotated data in a source domain (e.g., sunny weather) for achieving high performance on the test data in a target domain (e.g., rainy weather). Whether the target contains a single homogeneous domain or multiple heterogeneous domains, existing works always assume that there exist clear distinctions between the domains, which is often not true in practice (e.g., changes in weather). We study an open compound domain adaptation (OCDA) problem, in which the target is a compound of multiple homogeneous domains without domain l…
In this work, we present a novel upper bound of target error to address the problem for unsupervised domain adaptation. Recent studies reveal that a deep neural network can learn transferable features which generalize well to novel tasks. Furthermore, a theory proposed by Ben-David et al. (2010) provides a upper bound for target error when transferring the knowledge, which can be summarized as minimizing the source error and distance between marginal distributions simultaneously. However, common methods based on the theory usually ignore the joint error such that samples from different classes…
Korea Advanced Institute of Science and Technology (KAIST)（韩国科学技术院）
Unsupervised domain adaptation (UDA) for semantic segmentation has been attracting attention recently, as it could be beneficial for various label-scarce real-world scenarios (e.g., robot control, autonomous driving, medical imaging, etc.). Despite the significant progress in this field, current works mainly focus on a single-source single-target setting, which cannot handle more practical settings of multiple targets or even unseen targets. In this paper, we investigate open compound domain adaptation (OCDA), which deals with mixed and novel situations at the same time, for semantic segmentat…
The University of Queensland（昆士兰大学）；Peng Cheng Laboratory（鹏城实验室）；Bio-Computing Research Center, Harbin Institute of Technology（哈尔滨工业大学生物计算研究中心）
Domain adaptation techniques, which focus on adapting models between distributionally different domains, are rarely explored in the video recognition area due to the significant spatial and temporal shifts across the source (i.e. training) and target (i.e. test) domains. As such, recent works on visual domain adaptation which leverage adversarial learning to unify the source and target video representations and strengthen the feature transferability are not highly effective on the videos. To overcome this limitation, in this paper, we learn a domain-agnostic video classifier instead of learnin…
Huazhong University of Science and Technology（华中科技大学）
Unsupervised domain adaptation is critical in various computer vision tasks, such as object detection, instance segmentation, etc. They attempt to reduce domain bias-induced performance degradation while also promoting model application speed. Previous works in domain adaptation object detection attempt to align image-level and instance-level shifts to eventually minimize the domain discrepancy, but they may align single-class features to mixed-class features in image-level domain adaptation because each image in the object detection task may be more than one class and object. In order to achi…
Domain-invariant Mixed-domain Semi-supervised Medical Image Segmentation with Clustered Maximum Mean Discrepancy Alignment
Carnegie Mellon University（卡内基梅隆大学）；Ho Chi Minh University of Science（胡志明科学大学）；Northwestern University（西北大学）；University of North Carolina at Charlotte（北卡罗来纳州夏洛特大学）
This paper addresses text recognition for domains with limited manual annotations by a simple self-training strategy. Our approach should reduce human annotation effort when target domain data is plentiful, such as when transcribing a collection of single person's correspondence or a large manuscript. We propose to train a seed system on large scale data from related domains mixed with available annotated data from the target domain. The seed system transcribes the unannotated data from the target domain which is then used to train a better system. We study several confidence measures and even…
When is Test-Time Adaptation Identifiable From Unlabeled Evidence?
Nanyang Technological University（南洋理工大学）；Institute for Digital Molecular Analytics and Science（数字分子分析与科学研究所）；School of Electrical and Electronic Engineering（电气与电子工程学院）
Back to the Source: Diffusion-Driven Test-Time Adaptation
Shanghai Jiao Tong University（上海交通大学）；The University of Hong Kong（香港大学）；University of California, Berkeley（加州大学伯克利分校）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；DeepMind（深度思维）
MITA: Bridging the Gap between Model and Data for Test-time Adaptation
Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Pennsylvania State University（宾夕法尼亚州立大学）；Kuaishou Technology（快手科技）
LiDAR-based 3D object detection is an indispensable task in advanced autonomous driving systems. Though impressive detection results have been achieved by superior 3D detectors, they suffer from significant performance degeneration when facing unseen domains, such as different LiDAR configurations, different cities, and weather conditions. The mainstream approaches tend to solve these challenges by leveraging unsupervised domain adaptation (UDA) techniques. However, these UDA solutions just yield unsatisfactory 3D detection results when there is a severe domain shift, e.g., from Waymo (64-beam…
