We advocate the use of conformal prediction (CP) to enhance rule-based multi-label classification (MLC). In particular, we highlight the mutual benefit of CP and rule learning: Rules have the ability to provide natural (non-)conformity scores, which are required by CP, while CP suggests a way to calibrate the assessment of candidate rules, thereby supporting better predictions and more elaborate decision making. We illustrate the potential usefulness of calibrated conformity scores in a case study on lazy multi-label rule learning.
Conformal Prediction for Federated Uncertainty Quantification Under Label Shift
Ecole Polytechnique（巴黎综合理工学院）；Lagrange Mathematics and Computing Research Center（拉格朗日数学与计算研究中心）；Skolkovo Institute of Science and Technology（斯科尔科沃科学技术研究院）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）；Technology Innovation Institute（技术创新研究院）
CONFIDERAI: a novel CONFormal Interpretable-by-Design score function for Explainable and Reliable Artificial Intelligence
Institute of Electronics, Information Engineering and Telecommunications - National Research Council of Italy (CNR-IEIIT)（电子信息与电信研究所 - 意大利国家科研理事会（CNR-IEIIT））；Politecnico di Torino - Department of Control and Computer Engineering (DAUIN)（托尼诺理工学院 - 控制与计算机工程系（DAUIN））；Rulex Innovation Labs（Rulex创新实验室）
University of Southern California（南加州大学）；Zhejiang University（浙江大学）
This paper develops novel conformal methods to test whether a new observation was sampled from the same distribution as a reference set. Blending inductive and transductive conformal inference in an innovative way, the described methods can re-weight standard conformal p-values based on dependent side information from known out-of-distribution data in a principled way, and can automatically take advantage of the most powerful model from any collection of one-class and binary classifiers. The solution can be implemented either through sample splitting or via a novel transductive cross-validatio…
Conformal Inference for Open-Set and Imbalanced Classification
Department of Data Sciences and Operations, University of Southern California, USA（美国南加州大学数据科学与运营系）；Department of Statistics, University of California, Irvine, USA（美国加州大学伊尔弗斯分校统计系）；Dipartimento di Scienze Economico-Sociali e Matematico-Statistiche, Università di Torino and Collegio Carlo Alberto, Italy（意大利都灵大学经济与社会科学系和卡尔罗·阿尔贝托学院）；Department of Computer Science, University of Southern California, USA（美国南加州大学计算机科学系）
Conformal Prediction for Deep Classifier via Label Ranking
Southern University of Science and Technology（南方科技大学）；ShanghaiTech University（上海科技大学）；Rutgers University（罗格斯大学）；University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）；Chongqing University（重庆大学）
CPATTA: Conformal Supervision Allocation For Active Test-Time Adaptation
Computer Science and Engineering, University of California San Diego（加州大学圣地亚哥分校计算机科学与工程系）；New Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所模式识别新实验室）；College of Computer Science and Electronic Engineering, Hunan University（湖南大学计算机科学与电子工程学院）
Stable Localized Conformal Prediction via Transduction
School of Statistics；Data Science, Nankai University, Tianjin, China；School of Mathematics \& Statistics, The University of Melbourne, Melbourne, Australia
Center for Advanced Mathematics in Energy Research Applications（能源研究应用高级数学中心）；Synchrotron Infrared Structural Biology Program（同步辐射红外结构生物学计划）；Molecular Biophysics and Integrated Bioimaging Division（分子生物物理与整合生物成像部）
Exploiting dependencies between labels is considered to be crucial for multi-label classification. Rules are able to expose label dependencies such as implications, subsumptions or exclusions in a human-comprehensible and interpretable manner. However, the induction of rules with multiple labels in the head is particularly challenging, as the number of label combinations which must be taken into account for each rule grows exponentially with the number of available labels. To overcome this limitation, algorithms for exhaustive rule mining typically use properties such as anti-monotonicity or d…
INESCTEC；LIACS；HasLab, INESC TEC, Departamento de Informática, Universidade do Minho（Minho大学信息学院HasLab, INESC TEC）；DCC - Faculdade de Ciencias, Universidade do Porto（波尔图大学科学学院DCC）；LIACS, Leiden, Netherlands（莱顿大学LIACS）
In this paper we investigate two variants of association rules for preference data, Label Ranking Association Rules and Pairwise Association Rules. Label Ranking Association Rules (LRAR) are the equivalent of Class Association Rules (CAR) for the Label Ranking task. In CAR, the consequent is a single class, to which the example is expected to belong to. In LRAR, the consequent is a ranking of the labels. The generation of LRAR requires special support and confidence measures to assess the similarity of rankings. In this work, we carry out a sensitivity analysis of these similarity-based measur…
Unlike the typical classification setting where each instance is associated with a single class, in multi-label learning each instance is associated with multiple classes simultaneously. Therefore the learning task in this setting is to predict the subset of classes to which each instance belongs. This work examines the application of a recently developed framework called Conformal Prediction (CP) to the multi-label learning setting. CP complements the predictions of machine learning algorithms with reliable measures of confidence. As a result the proposed approach instead of just predicting t…
Recently, several authors have advocated the use of rule learning algorithms to model multi-label data, as rules are interpretable and can be comprehended, analyzed, or qualitatively evaluated by domain experts. Many rule learning algorithms employ a heuristic-guided search for rules that model regularities contained in the training data and it is commonly accepted that the choice of the heuristic has a significant impact on the predictive performance of the learner. Whereas the properties of rule learning heuristics have been studied in the realm of single-label classification, there is no su…
TU Darmstadt（达姆施塔特工业大学）；JKU Linz（林茨约翰内斯·开普勒大学）；Paderborn University（帕德博恩大学）
In multi-label classification, where the evaluation of predictions is less straightforward than in single-label classification, various meaningful, though different, loss functions have been proposed. Ideally, the learning algorithm should be customizable towards a specific choice of the performance measure. Modern implementations of boosting, most prominently gradient boosted decision trees, appear to be appealing from this point of view. However, they are mostly limited to single-label classification, and hence not amenable to multi-label losses unless these are label-wise decomposable. In t…
Technische Universität Darmstadt（达姆施塔特工业大学）；Universität Paderborn（帕德博恩大学）
Multi-label classification (MLC) is a supervised learning problem in which, contrary to standard multiclass classification, an instance can be associated with several class labels simultaneously. In this chapter, we advocate a rule-based approach to multi-label classification. Rule learning algorithms are often employed when one is not only interested in accurate predictions, but also requires an interpretable theory that can be understood, analyzed, and qualitatively evaluated by domain experts. Ideally, by revealing patterns and regularities contained in the data, a rule-based theory yields …
The Air Force Research Laboratory - Munitions Directorate（美国空军研究实验室弹药部）
In multi-label classification tasks, each problem instance is associated with multiple classes simultaneously. In such settings, the correlation between labels contains valuable information that can be used to obtain more accurate classification models. The correlation between labels can be exploited at different levels such as capturing the pair-wise correlation or exploiting the higher-order correlations. Even though the high-order approach is more capable of modeling the correlation, it is computationally more demanding and has scalability issues. This paper aims at exploiting the high-orde…
Conformal Prediction with Corrupted Labels: Uncertain Imputation and Robust Re-weighting
Department of Computer Science（计算机科学系）；Technion, Israel（技术学院，以色列）；Department of Electrical Engineering and Computer Science（电气工程与计算机科学系）；Massachusetts Institute of Technology（麻省理工学院）；Departments of Electrical and Computer Engineering and of Computer Science（电气工程与计算机科学系）
In many applications labeled data is not readily available, and needs to be collected via pain-staking human supervision. We propose a rule-exemplar method for collecting human supervision to combine the efficiency of rules with the quality of instance labels. The supervision is coupled such that it is both natural for humans and synergistic for learning. We propose a training algorithm that jointly denoises rules via latent coverage variables, and trains the model through a soft implication loss over the coverage and label variables. The denoised rules and trained model are used jointly for i…
Being able to model correlations between labels is considered crucial in multi-label classification. Rule-based models enable to expose such dependencies, e.g., implications, subsumptions, or exclusions, in an interpretable and human-comprehensible manner. Albeit the number of possible label combinations increases exponentially with the number of available labels, it has been shown that rules with multiple labels in their heads, which are a natural form to model local label dependencies, can be induced efficiently by exploiting certain properties of rule evaluation measures and pruning the lab…
