Partial Label Learning (PLL) aims to learn from the data where each training example is associated with a set of candidate labels, among which only one is correct. The key to deal with such problem is to disambiguate the candidate label sets and obtain the correct assignments between instances and their candidate labels. In this paper, we interpret such assignments as instance-to-label matchings, and reformulate the task of PLL as a matching selection problem. To model such problem, we propose a novel Graph Matching based Partial Label Learning (GM-PLL) framework, where Graph Matching (GM) scheme is incorporated owing to its excellent capability of exploiting the instance and label relationship. Meanwhile, since conventional one-to-one GM algorithm does not satisfy the constraint of PLL problem that multiple instances may correspond to the same label, we extend a traditional one-to-one probabilistic matching algorithm to the many-to-one constraint, and make the proposed framework accommodate to the PLL problem. Moreover, we also propose a relaxed matching prediction model, which can improve the prediction accuracy via GM strategy. Extensive experiments on both artificial and real-world data sets demonstrate that the proposed method can achieve superior or comparable performance against the state-of-the-art methods.
Learnability with Partial Labels and Adaptive Nearest Neighbors
Basque Center for Applied Mathematics（巴斯克应用数学中心）；Ikerbasque, Basque Foundation for Science（Ikerbasque，巴斯克科学基金会）；University of the Basque Country, Spain（巴斯克国家大学，西班牙）；University of California San Diego, USA（加州大学圣地亚哥分校，美国）
School of Computer Science and Engineering, Nanyang Technological University, Singapore（南洋理工大学计算机科学与工程学院）；School of Computer Science and Engineering, Southeast University（东南大学计算机科学与工程学院）；Department of Computer Science, Hong Kong Baptist University（香港浸会大学计算机科学系）；The University of Queensland, Australia（昆士兰大学）；Center for Advanced Intelligence Project, RIKEN（理化学研究所高级智能项目中心）；The University of Tokyo, Japan（东京大学）
Partial-label learning (PLL) is a multi-class classification problem, where each training example is associated with a set of candidate labels. Even though many practical PLL methods have been proposed in the last two decades, there lacks a theoretical understanding of the consistency of those methods-none of the PLL methods hitherto possesses a generation process of candidate label sets, and then it is still unclear why such a method works on a specific dataset and when it may fail given a different dataset. In this paper, we propose the first generation model of candidate label sets, and dev…
Shanghai Jiao Tong University（上海交通大学）；Shanghai AI Laboratory（上海人工智能实验室）
Real-world data usually couples the label ambiguity and heavy imbalance, challenging the algorithmic robustness of partial label learning (PLL) and long-tailed learning (LT). The straightforward combination of LT and PLL, i.e., LT-PLL, suffers from a fundamental dilemma: LT methods build upon a given class distribution that is unavailable in PLL, and the performance of PLL is severely influenced in long-tailed context. We show that even with the auxiliary of an oracle class prior, the state-of-the-art methods underperform due to an adverse fact that the constant rebalancing in LT is harsh to t…
RIKEN Center for Advanced Intelligence Project（理化学研究所先进智能项目中心）；Southeast University（东南大学）；Chongqing University（重庆大学）；The University of Queensland（昆士兰大学）；Nanyang Technological University（南洋理工大学）；The University of Tokyo（东京大学）
Partial-label learning (PLL) utilizes instances with PLs, where a PL includes several candidate labels but only one is the true label (TL). In PLL, identification-based strategy (IBS) purifies each PL on the fly to select the (most likely) TL for training; average-based strategy (ABS) treats all candidate labels equally for training and let trained models be able to predict TL. Although PLL research has focused on IBS for better performance, ABS is also worthy of study since modern IBS behaves like ABS in the beginning of training to prepare for PL purification and TL selection. In this paper,…
Tuning the Right Foundation Models is What you Need for Partial Label Learning
School of Computer Science and Engineering, Southeast University, Nanjing 210096, China（南京大学计算机科学与工程学院）；Key Lab. of Computer Network and Information Integration (Southeast University), MoE, China（计算机网络与信息集成重点实验室（东南大学））
ALIM: Adjusting Label Importance Mechanism for Noisy Partial Label Learning
University of Chinese Academy of Sciences（中国科学院大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Chongqing University（重庆大学）；Nanyang Technological University（南洋理工大学）；Tsinghua University（清华大学）
Partial label learning (PLL) aims to solve the problem where each training instance is associated with a set of candidate labels, one of which is the correct label. Most PLL algorithms try to disambiguate the candidate label set, by either simply treating each candidate label equally or iteratively identifying the true label. Nonetheless, existing algorithms usually treat all labels and instances equally, and the complexities of both labels and instances are not taken into consideration during the learning stage. Inspired by the successful application of self-paced learning strategy in machine…
SI Analytics Co. Ltd（SI分析有限公司）；UW--Madison（威斯康星大学麦迪逊分校）；GIST（光州科学技术院）
Partial label learning (PLL) is a class of weakly supervised learning where each training instance consists of a data and a set of candidate labels containing a unique ground truth label. To tackle this problem, a majority of current state-of-the-art methods employs either label disambiguation or averaging strategies. So far, PLL methods without such techniques have been considered impractical. In this paper, we challenge this view by revealing the hidden power of the oldest and naivest PLL method when it is instantiated with deep neural networks. Specifically, we show that, with deep neural n…
Partial label learning (PLL) is a typical weakly supervised learning problem, where each training example is associated with a set of candidate labels among which only one is true. Most existing PLL approaches assume that the incorrect labels in each training example are randomly picked as the candidate labels and model the generation process of the candidate labels in a simple way. However, these approaches usually do not perform as well as expected due to the fact that the generation process of the candidate labels is always instance-dependent. Therefore, it deserves to be modeled in a refin…
School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；NLPR, Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所模式识别国家重点实验室）
Partial-label learning (PLL) is an important branch of weakly supervised learning where the single ground truth resides in a set of candidate labels, while the research rarely considers the label imbalance. A recent study for imbalanced partial-Label learning proposed that the combinatorial challenge of partial-label learning and long-tail learning lies in matching between a decent marginal prior distribution with drawing the pseudo labels. However, we believe that even if the pseudo label matches the prior distribution, the tail classes will still be difficult to learn because the total weigh…
Partial label learning (PLL) is a typical weakly supervised learning problem, where each training example is associated with a set of candidate labels among which only one is true. Most existing PLL approaches assume that the incorrect labels in each training example are randomly picked as the candidate labels. However, this assumption is not realistic since the candidate labels are always instance-dependent. In this paper, we consider instance-dependent PLL and assume that each example is associated with a latent label distribution constituted by the real number of each label, representing th…
Zhejiang University（浙江大学）；Chongqing University（重庆大学）；RIKEN Center for Advanced Intelligence Project（理化学研究所先进智能项目中心）
Partial label learning (PLL) is an important problem that allows each training example to be labeled with a coarse candidate set, which well suits many real-world data annotation scenarios with label ambiguity. Despite the promise, the performance of PLL often lags behind the supervised counterpart. In this work, we bridge the gap by addressing two key research challenges in PLL -- representation learning and label disambiguation -- in one coherent framework. Specifically, our proposed framework PiCO consists of a contrastive learning module along with a novel class prototype-based label disam…
Partial Label Learning (PLL) aims to learn from the data where each training instance is associated with a set of candidate labels, among which only one is correct. Most existing methods deal with such problem by either treating each candidate label equally or identifying the ground-truth label iteratively. In this paper, we propose a novel PLL approach called HERA, which simultaneously incorporates the HeterogEneous Loss and the SpaRse and Low-rAnk procedure to estimate the labeling confidence for each instance while training the model. Specifically, the heterogeneous loss integrates the stre…
Southeast University（东南大学）；RIKEN Center for Advanced Intelligence Project（理化学研究所革新智能综合研究中心）；The University of Queensland（昆士兰大学）；Nanyang Technological University（南洋理工大学）；University of Tokyo（东京大学）
Partial-label learning (PLL) is a typical weakly supervised learning problem, where each training instance is equipped with a set of candidate labels among which only one is the true label. Most existing methods elaborately designed learning objectives as constrained optimizations that must be solved in specific manners, making their computational complexity a bottleneck for scaling up to big data. The goal of this paper is to propose a novel framework of PLL with flexibility on the model and optimization algorithm. More specifically, we propose a novel estimator of the classification risk, th…
Zhejiang University（浙江大学）；University of Wisconsin-Madison（威斯康星大学麦迪逊分校）；Chongqing University（重庆大学）；RIKEN（理化学研究所）
Partial-label learning (PLL) is a peculiar weakly-supervised learning task where the training samples are generally associated with a set of candidate labels instead of single ground truth. While a variety of label disambiguation methods have been proposed in this domain, they normally assume a class-balanced scenario that may not hold in many real-world applications. Empirically, we observe degenerated performance of the prior methods when facing the combinatorial challenge from the long-tailed distribution and partial-labeling. In this work, we first identify the major reasons that the prior…
Shandong University（山东大学）；Joint SDU-NTU Centre for Artificial Intelligence Research, Shandong University（山东大学山东大学-南洋理工大学人工智能联合研究院）；George Mason University（乔治梅森大学）
Partial-label learning (PLL) generally focuses on inducing a noise-tolerant multi-class classifier by training on overly-annotated samples, each of which is annotated with a set of labels, but only one is the valid label. A basic promise of existing PLL solutions is that there are sufficient partial-label (PL) samples for training. However, it is more common than not to have just few PL samples at hand when dealing with new tasks. Furthermore, existing few-shot learning algorithms assume precise labels of the support set; as such, irrelevant labels may seriously mislead the meta-learner and th…
Mitigating Instance Entanglement in Instance-Dependent Partial Label Learning
School of Computer Science and Technology, Xi’an Jiaotong University（西安交通大学计算机科学与技术学院）；Shaanxi Province Key Laboratory of Big Data Knowledge Engineering, Xi’an Jiaotong University（陕西省大数据知识工程重点实验室）；School of Distance Education, Xi’an Jiaotong University（西安交通大学继续教育学院）
Partial Label Learning (PLL) aims to train a classifier when each training instance is associated with a set of candidate labels, among which only one is correct but is not accessible during the training phase. The common strategy dealing with such ambiguous labeling information is to disambiguate the candidate label sets. Nonetheless, existing methods ignore the disambiguation difficulty of instances and adopt the single-trend training mechanism. The former would lead to the vulnerability of models to the false positive labels and the latter may arouse error accumulation problem. To remedy th…
Diffusion Disambiguation Models for Partial Label Learning
College of Computer Science and Technology, Qingdao University（青岛大学计算机科学与技术学院）；Department of Control Science and Engineering, College of Electronics and Information Engineering, Tongji University（电子信息工程学院控制科学与工程系，同济大学）；School of Electronic Information and Electrical Engineering, Shanghai Jiao Tong University（上海交通大学电子信息与电气工程学院）
AsyCo: An Asymmetric Dual-task Co-training Model for Partial-label Learning
College of Computer Science, Chongqing University（重庆大学计算机学院）；Institute of Software, Chinese Academy of Sciences（中国科学院软件研究所）；University of Chinese Academy of Sciences（中国科学院大学）；School of Software Technology, Zhejiang University（浙江大学软件学院）；Singapore University of Technology and Design（新加坡科技设计大学）
Exploiting the Potential Supervision Information of Clean Samples in Partial Label Learning
Department of Computer and Information Science, University of Macau（澳门大学计算机与信息科学系）；Department of Computer Science, Hong Kong Baptist University（香港 Baptist 大学计算机科学系）
Calibratable Disambiguation Loss for Multi-Instance Partial-Label Learning
School of Computer Science and Engineering, Southeast University（计算机科学与工程学院，东南大学）；Key Laboratory of Computer Network and Information Integration (Southeast University), MoE, China（计算机网络与信息集成重点实验室（东南大学），教育部，中国）；School of Information and Physical Sciences, The University of Newcastle（信息与物理科学学院，新castle大学）
College of Software Engineering, Southeast University（东南大学软件学院）；School of Computer Science and Engineering, Southeast University（东南大学计算机科学与工程学院）；Yam Pak Charitable Foundation School of Computing and Information Sciences, Saint Francis University（圣弗朗西斯大学叶荫堂计算与信息科学学院）；Department of Computer Science, City University of Hong Kong（香港城市大学计算机科学系）
Institut de Neurosciences de la Timone（蒂莫神经科学研究所）；Aix-Marseille Université（艾克斯-马赛大学）；CNRS（法国国家科学研究中心）；Institut Marseille Imaging（马赛成像研究所）；Laboratoire d’Informatique et Systèmes（计算机与系统实验室）
Population-wise matching of the cortical fold is necessary to identify biomarkers of neurological or psychiatric disorders. The difficulty comes from the massive interindividual variations in the morphology and spatial organization of the folds. This task is challenging at both methodological and conceptual levels. In the widely used registration-based techniques, these variations are considered as noise and the matching of folds is only implicit. Alternative approaches are based on the extraction and explicit identification of the cortical folds. In particular, representing cortical folding p…
Criteo AI Lab（Criteo人工智能实验室）；Ecole Polytechnique（巴黎综合理工学院）；INSA de Rouen Normandie（鲁昂国立应用科学学院）；Université Technologique de Compiègne（贡比涅技术大学）；Université de Rouen Normandie（鲁昂诺曼底大学）；Université de Bretagne Sud（南布列塔尼大学）
We address the problem of unsupervised domain adaptation under the setting of generalized target shift (joint class-conditional and label shifts). For this framework, we theoretically show that, for good generalization, it is necessary to learn a latent representation in which both marginals and class-conditional distributions are aligned across domains. For this sake, we propose a learning problem that minimizes importance weighted loss in the source domain and a Wasserstein distance between weighted marginals. For a proper weighting, we provide an estimator of target label proportion by blen…
University of Maryland（马里兰大学）；Texas A&M University（得克萨斯农工大学）；Johns Hopkins University（约翰斯·霍普金斯大学）
Given a collection of vertex-aligned networks and an additional label-shuffled network, we propose procedures for leveraging the signal in the vertex-aligned collection to recover the labels of the shuffled network. We consider matching the shuffled network to averages of the networks in the vertex-aligned collection at different levels of granularity. We demonstrate both in theory and practice that if the graphs come from different network classes, then clustering the networks into classes followed by matching the new graph to cluster-averages can yield higher fidelity matching performance th…
One of the fundamental tasks in understanding genomics is the problem of predicting Transcription Factor Binding Sites (TFBSs). With more than hundreds of Transcription Factors (TFs) as labels, genomic-sequence based TFBS prediction is a challenging multi-label classification task. There are two major biological mechanisms for TF binding: (1) sequence-specific binding patterns on genomes known as "motifs" and (2) interactions among TFs known as co-binding effects. In this paper, we propose a novel deep architecture, the Prototype Matching Network (PMN) to mimic the TF binding mechanisms. Our P…
Deep learning with noisy labels is challenging as deep neural networks have the high capacity to memorize the noisy labels. In this paper, we propose a learning algorithm called Co-matching, which balances the consistency and divergence between two networks by augmentation anchoring. Specifically, we have one network generate anchoring label from its prediction on a weakly-augmented image. Meanwhile, we force its peer network, taking the strongly-augmented version of the same image as input, to generate prediction close to the anchoring label. We then update two networks simultaneously by sele…
Microsoft Research Montreal（微软研究院蒙特利尔）；D. E. Shaw & Co.（D. E. Shaw公司）；UC Santa Barbara（加州大学圣巴巴拉分校）
Adversarial learning has demonstrated good performance in the unsupervised domain adaptation setting, by learning domain-invariant representations. However, recent work has shown limitations of this approach when label distributions differ between the source and target domains. In this paper, we propose a new assumption, generalized label shift ($GLS$), to improve robustness against mismatched label distributions. $GLS$ states that, conditioned on the label, there exists a representation of the input that is invariant between the source and target domains. Under $GLS$, we provide theoretical g…
Qiushi Academy for Advanced Studies, Zhejiang University（浙江大学求是高等研究院）；College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；State University of New York at Buffalo（纽约州立大学布法罗分校）；Zhejiang University School of Medicine（浙江大学医学院）
Domain adaptation is an important technique to alleviate performance degradation caused by domain shift, e.g., when training and test data come from different domains. Most existing deep adaptation methods focus on reducing domain shift by matching marginal feature distributions through deep transformations on the input features, due to the unavailability of target domain labels. We show that domain shift may still exist via label distribution shift at the classifier, thus deteriorating model performances. To alleviate this issue, we propose an approximate joint distribution matching scheme by…
Domain adaptation addresses the problem created when training data is generated by a so-called source distribution, but test data is generated by a significantly different target distribution. In this work, we present approximate label matching (ALM), a new unsupervised domain adaptation technique that creates and leverages a rough labeling on the test samples, then uses these noisy labels to learn a transformation that aligns the source and target samples. We show that the transformation estimated by ALM has favorable properties compared to transformations estimated by other methods, which do…
