While self-training has advanced semi-supervised semantic segmentation, it severely suffers from the long-tailed class distribution on real-world semantic segmentation datasets that make the pseudo-labeled data bias toward majority classes. In this paper, we present a simple and yet effective Distribution Alignment and Random Sampling (DARS) method to produce unbiased pseudo labels that match the true class distribution estimated from the labeled data. Besides, we also contribute a progressive data augmentation and labeling strategy to facilitate model training with pseudo-labeled data. Experiments on both Cityscapes and PASCAL VOC 2012 datasets demonstrate the effectiveness of our approach. Albeit simple, our method performs favorably in comparison with state-of-the-art approaches. Code will be available at https://github.com/CVMI-Lab/DARS.
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Nanjing Artificial Intelligence Research of IA（IA南京人工智能研究院）
Domain adaptive object detection (DAOD) assumes that both labeled source data and unlabeled target data are available for training, but this assumption does not always hold in real-world scenarios. Thus, source-free DAOD is proposed to adapt the source-trained detectors to target domains with only unlabeled target data. Existing source-free DAOD methods typically utilize pseudo labeling, where the performance heavily relies on the selection of confidence threshold. However, most prior works adopt a single fixed threshold for all classes to generate pseudo labels, which ignore the imbalanced cl…
University of Trento（特伦托大学）；Fondazione Bruno Kessler（布鲁诺·凯斯勒基金会）；Facebook Reality Labs（Facebook现实实验室）
Pseudo-LiDAR-based methods for monocular 3D object detection have received considerable attention in the community due to the performance gains exhibited on the KITTI3D benchmark, in particular on the commonly reported validation split. This generated a distorted impression about the superiority of Pseudo-LiDAR-based (PL-based) approaches over methods working with RGB images only. Our first contribution consists in rectifying this view by pointing out and showing experimentally that the validation results published by PL-based methods are substantially biased. The source of the bias resides in…
The Hong Kong Polytechnic University（香港理工大学）；Dalian University of Technology（大连理工大学）
Domain adaptive semantic segmentation aims to learn a model with the supervision of source domain data, and produce satisfactory dense predictions on unlabeled target domain. One popular solution to this challenging task is self-training, which selects high-scoring predictions on target samples as pseudo labels for training. However, the produced pseudo labels often contain much noise because the model is biased to source domain as well as majority categories. To address the above issues, we propose to directly explore the intrinsic pixel distributions of target domain data, instead of heavily…
MarvelOVD: Marrying Object Recognition and Vision-Language Models for Robust Open-Vocabulary Object Detection
School of Computer Science and Engineering, Sun Yat-sen University（中山大学计算机科学与工程学院）；Hefei University of Technology（合肥工业大学）；Tencent America（腾讯美国公司）；Dalian University of Technology（大连理工大学）；Peng Cheng Laboratory（鹏城实验室）；School of Computer Science and Engineering, National Engineering Research Center of Digital Life, Sun Yat-sen University（中山大学国家数字生活工程研究中心计算机科学与工程学院）
Weakly supervised semantic segmentation (WSSS) aims to produce pixel-wise class predictions with only image-level labels for training. To this end, previous methods adopt the common pipeline: they generate pseudo masks from class activation maps (CAMs) and use such masks to supervise segmentation networks. However, it is challenging to derive comprehensive pseudo masks that cover the whole extent of objects due to the local property of CAMs, i.e., they tend to focus solely on small discriminative object parts. In this paper, we associate the locality of CAMs with the texture-biased property of…
Semi-supervised semantic segmentation focuses on the exploration of a small amount of labeled data and a large amount of unlabeled data, which is more in line with the demands of real-world image understanding applications. However, it is still hindered by the inability to fully and effectively leverage unlabeled images. In this paper, we reveal that cross-window consistency (CWC) is helpful in comprehensively extracting auxiliary supervision from unlabeled data. Additionally, we propose a novel CWC-driven progressive learning framework to optimize the deep network by mining weak-to-strong con…
3D object detection networks tend to be biased towards the data they are trained on. Evaluation on datasets captured in different locations, conditions or sensors than that of the training (source) data results in a drop in model performance due to the gap in distribution with the test (or target) data. Current methods for domain adaptation either assume access to source data during training, which may not be available due to privacy or memory concerns, or require a sequence of lidar frames as an input. We propose a single-frame approach for source-free, unsupervised domain adaptation of lidar…
Carnegie Mellon University（卡内基梅隆大学）；University of California Berkeley（加州大学伯克利分校）
While numerous 3D detection works leverage the complementary relationship between RGB images and point clouds, developments in the broader framework of semi-supervised object recognition remain uninfluenced by multi-modal fusion. Current methods develop independent pipelines for 2D and 3D semi-supervised learning despite the availability of paired image and point cloud frames. Observing that the distinct characteristics of each sensor cause them to be biased towards detecting different objects, we propose DetMatch, a flexible framework for joint semi-supervised learning on 2D and 3D modalities…
Bridging the Gap between Labeled and Unlabeled Data via Unified Flow with Feature Memory Bank
City University of Macau（澳门城市大学）；Southeast University（东南大学）；Ocean University of China（中国海洋大学）；Nantes Université（南特大学）；Ecole Centrale Nantes（南特中央理工学院）；CAPACITES SAS；CNRS（法国国家科学研究中心）；LS2N（南特数字科学实验室）
Beijing Institute of Technology（北京理工大学）；Tsinghua University（清华大学）；Inceptio Technology（嬴彻科技）
Self-training has greatly facilitated domain adaptive semantic segmentation, which iteratively generates pseudo labels on unlabeled target data and retrains the network. However, realistic segmentation datasets are highly imbalanced, pseudo labels are typically biased to the majority classes and basically noisy, leading to an error-prone and suboptimal model. In this paper, we propose a simple region-based active learning approach for semantic segmentation under a domain shift, aiming to automatically query a small partition of image regions to be labeled while maximizing segmentation performa…
The capability of the traditional semi-supervised learning (SSL) methods is far from real-world application due to severely biased pseudo-labels caused by (1) class imbalance and (2) class distribution mismatch between labeled and unlabeled data. This paper addresses such a relatively under-explored problem. First, we propose a general pseudo-labeling framework that class-adaptively blends the semantic pseudo-label from a similarity-based classifier to the linear one from the linear classifier, after making the observation that both types of pseudo-labels have complementary properties in terms…
Tsinghua University（清华大学）；BNRist（北京国家信息科学与技术研究中心）；Hangzhou Zhuoxi Institute of Brain and Intelligence（杭州卓识脑与智能研究院）；OPPO Research Institute（OPPO研究院）；Beijing University of Posts and Telecommunications（北京邮电大学）
Active learning selects informative samples for annotation within budget, which has proven efficient recently on object detection. However, the widely used active detection benchmarks conduct image-level evaluation, which is unrealistic in human workload estimation and biased towards crowded images. Furthermore, existing methods still perform image-level annotation, but equally scoring all targets within the same image incurs waste of budget and redundant labels. Having revealed above problems and limitations, we introduce a box-level active detection framework that controls a box-based budget…
Active learning (AL) aims to minimize labeling efforts for data-demanding deep neural networks (DNNs) by selecting the most representative data points for annotation. However, currently used methods are ill-equipped to deal with biased data. The main motivation of this paper is to consider a realistic setting for pool-based semi-supervised AL, where the unlabeled collection of train data is biased. We theoretically derive an optimal acquisition function for AL in this setting. It can be formulated as distribution shift minimization between unlabeled train data and weakly-labeled validation dat…
PO3AD: Predicting Point Offsets toward Better 3D Point Cloud Anomaly Detection
School of Advanced Technology, Xi’an Jiaotong-Liverpool University（西交利物浦大学先进技术学院）；School of Electrical Engineering, Electronics and Computer Science, University of Liverpool（利物浦大学电气工程、电子与计算机科学学院）；Data Science Research Center, Duke Kunshan University（昆山杜克大学数据科学研究中心）
LCGC: Learning from Consistency Gradient Conflicting for Class-Imbalanced Semi-Supervised Debiasing
School of Software Engineering, Beijing Jiaotong University（北京交通大学软件学院）；School of Automation, Northwestern Polytechnical University（西北工业大学自动化学院）；School of Computing, Newcastle University（纽卡斯尔大学计算机学院）；School of Science, Chongqing University of Posts and Telecommunications（重庆邮电大学理学院）
Studies have proven that domain bias and label bias exist in different Facial Expression Recognition (FER) datasets, making it hard to improve the performance of a specific dataset by adding other datasets. For the FER bias issue, recent researches mainly focus on the cross-domain issue with advanced domain adaption algorithms. This paper addresses another problem: how to boost FER performance by leveraging cross-domain datasets. Unlike the coarse and biased expression label, the facial Action Unit (AU) is fine-grained and objective suggested by psychological studies. Motivated by this, we res…
International Institute of Information Technology, Hyderabad（海得拉巴国际信息技术学院）；Mercedes-Benz Research and Development India（梅赛德斯-奔驰印度研发中心）
One of the primary challenges in Semi-supervised Domain Adaptation (SSDA) is the skewed ratio between the number of labeled source and target samples, causing the model to be biased towards the source domain. Recent works in SSDA show that aligning only the labeled target samples with the source samples potentially leads to incomplete domain alignment of the target domain to the source domain. In our approach, to align the two domains, we leverage contrastive losses to learn a semantically meaningful and a domain agnostic feature space using the supervised samples from both domains. To mitigat…
Zhejiang University（浙江大学）；Westlake University（西lake大学）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；Hunan University（湖南大学）；Zhejiang University Shanghai Innovation Institute（浙江大学上海创新研究院）；The University of Texas at Austin（德克萨斯大学奥斯汀分校）
CLAF: Contrastive Learning with Augmented Features for Imbalanced Semi-Supervised Learning
National Key Laboratory for Novel Software Technology, Nanjing University（南京大学计算机软件新技术国家重点实验室）；School of Artificial Intelligence, Nanjing University（南京大学人工智能学院）
Fairness Without Labels: Pseudo-Balancing for Bias Mitigation in Face Gender Classification
National Institute of Informatics, Japan（日本信息机构国家研究所）；INESC-ID, Instituto Superior Técnico, University of Lisbon, Portugal（里斯本大学技术高等学院、INESC-ID，葡萄牙）；University of Amsterdam, The Netherlands（荷兰阿姆斯特丹大学）；LIMSI, CNRS / Université Paris-Saclay, France（法国CNRS/巴黎萨克雷大学LIMSI）
The Chinese University of Hong Kong（香港中文大学）；The Hong Kong University of Science and Technology（香港科技大学）
Deep learning models were frequently reported to learn from shortcuts like dataset biases. As deep learning is playing an increasingly important role in the modern healthcare system, it is of great need to combat shortcut learning in medical data as well as develop unbiased and trustworthy models. In this paper, we study the problem of developing debiased chest X-ray diagnosis models from the biased training data without knowing exactly the bias labels. We start with the observations that the imbalance of bias distribution is one of the key reasons causing shortcut learning, and the dataset bi…
National University of Singapore（新加坡国立大学）；Sea AI Lab（Sea AI 实验室）
Domain adaptation (DA) aims to transfer knowledge from a label-rich but heterogeneous domain to a label-scare domain, which alleviates the labeling efforts and attracts considerable attention. Different from previous methods focusing on learning domain-invariant feature representations, some recent methods present generic semi-supervised learning (SSL) techniques and directly apply them to DA tasks, even achieving competitive performance. One of the most popular SSL techniques is pseudo-labeling that assigns pseudo labels for each unlabeled data via the classifier trained by labeled data. Howe…
ULFine: Unbiased Lightweight Fine-tuning for Foundation-Model-Assisted Long-Tailed Semi-Supervised Learning
MIIT Key Laboratory of Pattern Analysis and Machine Intelligence, China（信息产业部模式分析与机器智能重点实验室，中国）；College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics（南京航空航天大学计算机科学与技术学院）
Beihang University（北京航空航天大学）；University of Technology Sydney（悉尼科技大学）；IIAI（起源人工智能研究院）；Tencent AI Lab（腾讯人工智能实验室）；Inception Institute of Artificial Intelligence（起源人工智能研究院）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）
Unsupervised domain adaptation (UDA) for person re-identification is challenging because of the huge gap between the source and target domain. A typical self-training method is to use pseudo-labels generated by clustering algorithms to iteratively optimize the model on the target domain. However, a drawback to this is that noisy pseudo-labels generally cause trouble in learning. To address this problem, a mutual learning method by dual networks has been developed to produce reliable soft labels. However, as the two neural networks gradually converge, their complementarity is weakened and they …
Tongji University（同济大学）；The University of Sydney（悉尼大学）；University of Science and Technology of China（中国科学技术大学）；University of Science and Technology Beijing（北京科技大学）；JD Explore Academy（京东探索研究院）
Source-free Domain Adaptation (SFDA) aims to adapt a pre-trained source model to the unlabeled target domain without accessing the well-labeled source data, which is a much more practical setting due to the data privacy, security, and transmission issues. To make up for the absence of source data, most existing methods introduced feature prototype based pseudo-labeling strategies to realize self-training model adaptation. However, feature prototypes are obtained by instance-level predictions based feature clustering, which is category-biased and tends to result in noisy labels since the visual…
Vox-UDA: Voxel-wise Unsupervised Domain Adaptation for Cryo-Electron Subtomogram Segmentation with Denoised Pseudo Labeling
School of Computing and Information Technology, University of Wollongong, Australia（澳大利亚卧龙岗大学计算与信息技术学院）；Ray and Stephanie Lane Computational Biology Department, Carnegie Mellon University, USA（美国卡内基梅隆大学莱恩计算生物学系）；Centre for Nutrition and Food Sciences, The University of Queensland, Australia（澳大利亚昆士兰大学营养与食品科学中心）；School of Electrical and Information Engineering, University of Sydney, Australia（澳大利亚悉尼大学电气与信息工程学院）；Department of Business Strategy and Innovation, Griffith University, Australia（澳大利亚格里菲斯大学商业战略与创新系）；Department of Biological Sciences, Purdue University, USA（美国普渡大学生物科学系）；NVIDIA, USA（美国英伟达公司）
Source-Free Domain Adaptation for Geospatial Point Cloud Semantic Segmentation
Aerospace Information Research Institute, Chinese Academy of Sciences（中国科学院航天信息研究所）；International Research Center of Big Data for Sustainable Development Goals（可持续发展目标大数据国际研究中心）；University of Chinese Academy of Sciences（中国科学院大学）；Zhengzhou Institute for Advanced Research of Henan Polytechnic University（河南理工大学郑州研究院）；Henan Polytechnic University（河南理工大学）；School of Aeronautic Engineering, Changsha University of Science and Technology（长沙理工大学航空工程学院）；China University of Geosciences, Beijing（中国地质大学（北京））
Peking University（北京大学）；National University of Singapore（新加坡国立大学）；Huawei Noah’s Ark Lab（华为诺亚方舟实验室）
Object detection for autonomous vehicles has received increasing attention in recent years, where labeled data are often expensive while unlabeled data can be collected readily, calling for research on semi-supervised learning for this area. Existing semi-supervised object detection (SSOD) methods usually assume that the labeled and unlabeled data come from the same data distribution. In autonomous driving, however, data are usually collected from different scenarios, such as different weather conditions or different times in a day. Motivated by this, we study a novel but challenging domain in…
Semi-Supervised Learning (SSL) is fundamentally a missing label problem, in which the label Missing Not At Random (MNAR) problem is more realistic and challenging, compared to the widely-adopted yet naive Missing Completely At Random assumption where both labeled and unlabeled data share the same class distribution. Different from existing SSL solutions that overlook the role of "class" in causing the non-randomness, e.g., users are more likely to label popular classes, we explicitly incorporate "class" into SSL. Our method is three-fold: 1) We propose Class-Aware Propensity (CAP) that exploit…
Exploring Beyond Logits: Hierarchical Dynamic Labeling Based on Embeddings for Semi-Supervised Classification
Key Laboratory of Intelligent Perception and Image Understanding of the Ministry of Education（教育部智能感知与图像理解重点实验室）；International Research Center for Intelligent Perception and Computation（国际智能感知与计算研究中心）；School of Artificial Intelligence, Xidian University（西安电子科技大学人工智能学院）
