Modern image classifiers are very accurate, but the predictions come without uncertainty estimates. Conformal predictors provide uncertainty estimates by computing a set of classes containing the correct class with a user-specified probability based on the classifier's probability estimates. To provide such sets, conformal predictors often estimate a cutoff threshold for the probability estimates based on a calibration set. Conformal predictors guarantee reliability only when the calibration set is from the same distribution as the test set. Therefore, conformal predictors need to be recalibrated for new distributions. However, in practice, labeled data from new distributions is rarely available, making calibration infeasible. In this work, we consider the problem of predicting the cutoff threshold for a new distribution based on unlabeled examples. While it is impossible in general to guarantee reliability when calibrating based on unlabeled examples, we propose a method that provides excellent uncertainty estimates under natural distribution shifts, and provably works for a specific model of a distribution shift.
Augmented prediction of a true class for Positive Unlabeled data under selection bias
Institute of Computer Science, Polish Academy of Sciences（波兰科学院计算机科学研究所）；Faculty of Mathematics and Information Science, Warsaw University of Technology（华沙理工大学数学与信息科学学院）
Minimax-optimal semi-supervised regression on unknown manifolds
Weizmann Institute of Science（魏茨曼科学研究所）
我们考虑半监督回归时，预测变量是从一个未知的流形。解决这个问题的一个简单的两步方法是：（i）使用标记和未标记的实例来估计任何一对点之间的流形测地线距离;以及（ii）基于这些距离估计应用k最近邻回归量。我们证明了足够多的未标记的点，这种简单的方法测地线kNN回归达到最佳的有限样本极大极小界的均方误差，如果已知的流形。此外，我们展示了如何可以有效地实现这种方法，只需要O（k N log N）的操作来估计所有N个标记和未标记点的回归函数。我们在两个具有流形结构的数据集上说明了这种方法：使用WiFi指纹的室内定位和面部姿态估计。在这两种情况下，测地线kNN比流行的拉普拉斯特征向量回归更准确，更快。
TILT: Target-induced loss tilting under covariate shift
Lab for Information and Decision Systems（信息与决策系统实验室）；Statistics and Data Science Center（统计与数据科学中心）；EECS, Massachusetts Institute of Technology（麻省理工学院电子工程与计算机科学系）；Mathematics and EECS, Massachusetts Institute of Technology（数学与电子工程与计算机科学系, 麻省理工学院）
AdaShadow: Responsive Test-time Model Adaptation in Non-stationary Mobile Environments
Northwestern Polytechnical University（西北工业大学）；City University of Hong Kong（香港城市大学）；The Hong Kong University of Science and Technology（香港科技大学）；Harbin Engineering University（哈尔滨工程大学）
