我们提出了一种名为 CHATTY（Coupled Holistic Adversarial Transport Terms with Yield，带让步的耦合整体对抗迁移项）的新技术，用于无监督域适应（UDA）。对抗训练通常被用于通过反转来自域判别器头的梯度来训练神经网络的特征提取器层，从而学习域不变表示。我们对对抗头、其训练目标以及分类器头提出了重大修改。为减少类别混淆，我们引入了一个子网络，该子网络以可学习的方式移动源域和目标域样本的分类器输出。我们使用一种新颖的迁移损失来控制这种移动，该损失使各类别簇彼此远离，并使分类器更容易为源域和目标域找到决策边界。将这种新损失加入到精心挑选的先前提出的损失组合中后，在基准数据集上的 UDA 结果相较于先前的最先进方法取得了提升。我们通过消融研究以及对目标域样本在表示空间中移动的可视化，展示了所提出损失项的重要性。
英文摘要：
We propose a new technique called CHATTY: Coupled Holistic Adversarial Transport Terms with Yield for Unsupervised Domain Adaptation. Adversarial training is commonly used for learning domain-invariant representations by reversing the gradients from a domain discriminator head to train the feature extractor layers of a neural network. We propose significant modifications to the adversarial head, its training objective, and the classifier head. With the aim of reducing class confusion, we introduce a sub-network which displaces the classifier outputs of the source and target domain samples in a learnable manner. We control this movement using a novel transport loss that spreads class clusters away from each other and makes it easier for the classifier to find the decision boundaries for both the source and target domains. The results of adding this new loss to a careful selection of previously proposed losses leads to improvement in UDA results compared to the previous state-of-the-art methods on benchmark datasets. We show the importance of the proposed loss term using ablation studies and visualization of the movement of target domain sample in representation space.
Human skin detection in images is a widely studied topic of Computer Vision for which it is commonly accepted that analysis of pixel color or local patches may suffice. This is because skin regions appear to be relatively uniform and many argue that there is a small chromatic variation among different samples. However, we found that there are strong biases in the datasets commonly used to train or tune skin detection methods. Furthermore, the lack of contextual information may hinder the performance of local approaches. In this paper we present a comprehensive evaluation of holistic and local …
Efficient Long-Short Temporal Attention Network for Unsupervised Video Object Segmentation
School of Computer Science and Technology, Hangzhou Dianzi University（杭州电子科技大学计算机科学与技术学院）；Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ)（广东省人工智能与数字经济实验室（深圳））；School of Electrical and Computer Engineering, Peking University（北京大学电气与计算机工程学院）；College of Systems Engineering, National University of Defense Technology（国防科技大学系统工程学院）；College of Computer Science, Zhejiang University（浙江大学计算机学院）
Seoul National University（首尔大学）；University of California at Merced（加州大学默塞德分校）
We introduce a new problem of generating an image based on a small number of key local patches without any geometric prior. In this work, key local patches are defined as informative regions of the target object or scene. This is a challenging problem since it requires generating realistic images and predicting locations of parts at the same time. We construct adversarial networks to tackle this problem. A generator network generates a fake image as well as a mask based on the encoder-decoder framework. On the other hand, a discriminator network aims to detect fake images. The network is train…
University of Southern California（南加州大学）；Baidu Research（百度研究）；National Engineering Laboratory for Deep Learning Technology and Applications（深度学习技术及应用国家工程实验室）
Learning to estimate 3D geometry in a single image by watching unlabeled videos via deep convolutional network has made significant process recently. Current state-of-the-art (SOTA) methods, are based on the learning framework of rigid structure-from-motion, where only 3D camera ego motion is modeled for geometry estimation.However, moving objects also exist in many videos, e.g. moving cars in a street scene. In this paper, we tackle such motion by additionally incorporating per-pixel 3D object motion into the learning framework, which provides holistic 3D scene flow understanding and helps si…
A Holistic Method for Superquadric Fitting Using Unsupervised Clustering Analysis
State Key Laboratory of Mathematical Sciences, Academy of Mathematics and Systems Science, Chinese Academy of Sciences（数学科学国家重点实验室，数学与系统科学学院，中国科学院）；University of Chinese Academy of Sciences（中国科学院大学）；Robotics Institute, School of Mechanical Engineering and Automation, Beihang University（北京航空航天大学机械工程与自动化学院机器人研究所）
The University of Hong Kong（香港大学）；The Chinese University of Hong Kong（香港中文大学）；SenseTime Research（商汤科技研究院）
In this paper, we present a self-training method, named ST3D++, with a holistic pseudo label denoising pipeline for unsupervised domain adaptation on 3D object detection. ST3D++ aims at reducing noise in pseudo label generation as well as alleviating the negative impacts of noisy pseudo labels on model training. First, ST3D++ pre-trains the 3D object detector on the labeled source domain with random object scaling (ROS) which is designed to reduce target domain pseudo label noise arising from object scale bias of the source domain. Then, the detector is progressively improved through alternati…
Domain-Shared Learning and Gradual Alignment for Unsupervised Domain Adaptation Visible-Infrared Person Re-Identification
State Key Laboratory of Electromechanical Integrated Manufacturing of High-Performance Electronic Equipments, Xidian University, Xi’an, Shaanxi 710071, China（高性能电子装备机电一体化制造国家重点实验室，西安电子科技大学，陕西西安710071，中国）；Center for Complex Systems, School of Mechano-Electronic Engineering, Xidian University, Xi’an, Shaanxi 710071, China（复杂系统中心，机电工程学院，西安电子科技大学，陕西西安710071，中国）
Dalian University of Technology（大连理工大学）；Data Storage and Intelligent Vision Technical Research Dept Huawei Cloud（华为云数据存储与智能视觉技术研究部）；Key Lab of Machine Perception Peking University（北京大学机器感知重点实验室）
Data-driven based approaches, in spite of great success in many tasks, have poor generalization when applied to unseen image domains, and require expensive cost of annotation especially for dense pixel prediction tasks such as semantic segmentation. Recently, both unsupervised domain adaptation (UDA) from large amounts of synthetic data and semi-supervised learning (SSL) with small set of labeled data have been studied to alleviate this issue. However, there is still a large gap on performance compared to their supervised counterparts. We focus on a more practical setting of semi-supervised do…
Tongji University（同济大学）；The University of Adelaide（阿德莱德大学）
Unsupervised cross-domain person re-identification (Re-ID) aims to adapt the information from the labelled source domain to an unlabelled target domain. Due to the lack of supervision in the target domain, it is crucial to identify the underlying similarity-and-dissimilarity relationships among the unlabelled samples in the target domain. In order to use the whole data relationships efficiently in mini-batch training, we apply a series of memory modules to maintain an up-to-date representation of the entire dataset. Unlike the simple exemplar memory in previous works, we propose a novel multi-…
Sketches make an intuitive and powerful visual expression as they are fast executed freehand drawings. We present a method for synthesizing realistic photos from scene sketches. Without the need for sketch and photo pairs, our framework directly learns from readily available large-scale photo datasets in an unsupervised manner. To this end, we introduce a standardization module that provides pseudo sketch-photo pairs during training by converting photos and sketches to a standardized domain, i.e. the edge map. The reduced domain gap between sketch and photo also allows us to disentangle them i…
We present a novel approach to perform the unsupervised domain adaptation for object detection through forward-backward cyclic (FBC) training. Recent adversarial training based domain adaptation methods have shown their effectiveness on minimizing domain discrepancy via marginal feature distributions alignment. However, aligning the marginal feature distributions does not guarantee the alignment of class conditional distributions. This limitation is more evident when adapting object detectors as the domain discrepancy is larger compared to the image classification task, e.g. various number of …
In this work, we introduce panoramic panoptic segmentation, as the most holistic scene understanding, both in terms of Field of View (FoV) and image-level understanding for standard camera-based input. A complete surrounding understanding provides a maximum of information to a mobile agent. This is essential information for any intelligent vehicle to make informed decisions in a safety-critical dynamic environment such as real-world traffic. In order to overcome the lack of annotated panoramic images, we propose a framework which allows model training on standard pinhole images and transfers t…
Metadata-Driven Federated Learning of Connectional Brain Templates in Non-IID Multi-Domain Scenarios
National Engineering Laboratory for Integrated Aero-Space-Ground-Ocean Big Data Application Technology, School of Computer Science and Engineering, Northwestern Polytechnical University（西北工业大学计算机学院航空航天地海一体化大数据应用技术国家工程实验室）；Imperial College London（帝国理工学院）
Beihang University（北京航空航天大学）；Shenzhen Institutes of Advanced Technology, Chinese Academy of Science（中国科学院深圳先进技术研究院）；SenseTime Research（商汤科技研究院）；Shanghai AI Laboratory（上海人工智能实验室）
Domain adaptive object detection (DAOD) is a promising way to alleviate performance drop of detectors in new scenes. Albeit great effort made in single source domain adaptation, a more generalized task with multiple source domains remains not being well explored, due to knowledge degradation during their combination. To address this issue, we propose a novel approach, namely target-relevant knowledge preservation (TRKP), to unsupervised multi-source DAOD. Specifically, TRKP adopts the teacher-student framework, where the multi-head teacher network is built to extract knowledge from labeled sou…
HierDAMap: Towards Universal Domain Adaptive BEV Mapping via Hierarchical Perspective Priors
National Engineering Research Center of Robot Visual Perception and Control Technology, Hunan University（湖南大学国家机器人视觉感知与控制工程研究中心）；State Key Laboratory of Extreme Photonics and Instrumentation, Zhejiang University（浙江大学极端光子学与仪器科学国家重点实验室）；College of Computer Science and Electronic Engineering, Hunan University（湖南大学计算机科学与电子工程学院）；Hunan Vanguard Group Corporation Limited（湖南先锋集团有限公司）
