In this work, we study some novel applications of conformal inference techniques to the problem of providing machine learning procedures with more transparent, accurate, and practical performance guarantees. We provide a natural extension of the traditional conformal prediction framework, done in such a way that we can make valid and well-calibrated predictive statements about the future performance of arbitrary learning algorithms, when passed an as-yet unseen training set. In addition, we include some nascent empirical examples to illustrate potential applications.
Conformal prediction is a technique for constructing prediction intervals that attain valid coverage in finite samples, without making distributional assumptions. Despite this appeal, existing conformal methods can be unnecessarily conservative because they form intervals of constant or weakly varying length across the input space. In this paper we propose a new method that is fully adaptive to heteroscedasticity. It combines conformal prediction with classical quantile regression, inheriting the advantages of both. We establish a theoretical guarantee of valid coverage, supplemented by extens…
Probabilistic Conformal Prediction with Approximate Conditional Validity
Lagrange Mathematics and Computing Research Center（拉格朗日数学与计算研究中心）；Skolkovo Institute of Science and Technology（斯科尔科沃科学技术研究所）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）；Ecole Polytechnique（巴黎综合理工学院）
Reliable Real-Time Value at Risk Estimation via Quantile Regression Forest with Conformal Calibration
Institute of Statistics and Big Data, Renmin University of China（统计与大数据研究院，中国人民大学）；Department of Decision Analytics and Operations, City University of Hong Kong（决策分析与运营部门，香港城市大学）；School of Information, Renmin University of China（信息学院，中国人民大学）
Decision-making is critical at each stage of drug development and making informed and transparent Go/No-Go decisions require a sound quantitative decision framework. We designed and implemented GOahead, a comprehensive web-based tool to improve how statisticians and collaborators could prospectively plan and implement the selected Go/No-Go decision approach in real-time. In the paper, we conducted a comprehensive overview of dual-criterion and confidence interval-based approaches to enable quantitative decision-making. illustrative examples are demonstrated for single and two arms designs in b…
We investigate a long-debated question, which is how to create predictive models of recidivism that are sufficiently accurate, transparent, and interpretable to use for decision-making. This question is complicated as these models are used to support different decisions, from sentencing, to determining release on probation, to allocating preventative social services. Each use case might have an objective other than classification accuracy, such as a desired true positive rate (TPR) or false positive rate (FPR). Each (TPR, FPR) pair is a point on the receiver operator characteristic (ROC) curve…
Fraunhofer Institute for Optronics, System Technologies, and Image Exploitation IOSB（弗劳恩霍夫光电、系统技术与图像利用研究所IOSB）；University of Stuttgart（斯图加特大学）；Fraunhofer Institute for Manufacturing Engineering and Automation IPA（弗劳恩霍夫制造工程与自动化研究所IPA）
Predictions obtained by, e.g., artificial neural networks have a high accuracy but humans often perceive the models as black boxes. Insights about the decision making are mostly opaque for humans. Particularly understanding the decision making in highly sensitive areas such as healthcare or fifinance, is of paramount importance. The decision-making behind the black boxes requires it to be more transparent, accountable, and understandable for humans. This survey paper provides essential definitions, an overview of the different principles and methodologies of explainable Supervised Machine Lear…
National Technical University of Athens（雅典国立技术大学）；GRNET National Infrastructures for Research and Technology（GRNET 国家研究与技术基础设施）；University of Lincoln（林肯大学）
The paper presents a novel deep learning approach, which extracts latent information from trained Deep Neural Networks (DNNs) and derives concise representations that are analyzed in an effective, unified way for prediction purposes. It is well known that DNNs are capable of analyzing complex data; however, they lack transparency in their decision making, in the sense that it is not straightforward to justify their prediction, or to visualize the features on which the decision was based. Moreover, they generally require large amounts of data in order to learn and become able to adapt to differ…
