arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2202.07650cs.LG
收藏

具有有限假阳性的共形预测集

Conformal Prediction Sets with Limited False Positives

CSAIL, Massachusetts Institute of Technology(麻省理工学院计算机科学与人工智能实验室);Google Research(谷歌研究院)

Adam Fisch, Tal Schuster, Tommi Jaakkola, Regina Barzilay

摘要:

我们开发了一种新的多标签共形预测方法,我们的目标是输出一组精确的有前途的预测候选人与有限数量的不正确的答案。标准保形预测通过构建校准的候选集来代替单个预测,从而提供适应模型不确定性的能力,并保证该集合包含具有高概率的正确答案。然而,为了服从这种覆盖性质,共形集可能会被嘈杂的候选者淹没-这可能使它们在实践中毫无帮助。这与预算有限的实际应用特别相关,并且与假阳性相关联的成本(货币或其他)不可忽略。我们建议通过强制在预测的共形集合中存在不正确的候选者(即,假阳性的总数)根据用户指定的容限来限定。受此约束,我们的算法然后优化集合覆盖的广义概念(即,真阳性率),其允许给定查询的任何数量的真答案(包括零)。我们在自然语言处理、计算机视觉和计算化学中的许多分类任务中证明了这种方法的有效性。

英文摘要:

We develop a new approach to multi-label conformal prediction in which we aim to output a precise set of promising prediction candidates with a bounded number of incorrect answers. Standard conformal prediction provides the ability to adapt to model uncertainty by constructing a calibrated candidate set in place of a single prediction, with guarantees that the set contains the correct answer with high probability. In order to obey this coverage property, however, conformal sets can become inundated with noisy candidates -- which can render them unhelpful in practice. This is particularly relevant to practical applications where there is a limited budget, and the cost (monetary or otherwise) associated with false positives is non-negligible. We propose to trade coverage for a notion of precision by enforcing that the presence of incorrect candidates in the predicted conformal sets (i.e., the total number of false positives) is bounded according to a user-specified tolerance. Subject to this constraint, our algorithm then optimizes for a generalized notion of set coverage (i.e., the true positive rate) that allows for any number of true answers for a given query (including zero). We demonstrate the effectiveness of this approach across a number of classification tasks in natural language processing, computer vision, and computational chemistry.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑