从误分类到异常值：分类中的联合可靠性评估
From Misclassifications to Outliers: Joint Reliability Assessment in Classification
AI总结：
本文提出联合评估框架和双评分函数，提升分类器可靠性，适用于不同场景的稳健模型部署。
摘要：
构建可靠的分类器是将机器学习应用于现实世界应用中的基本挑战。一个可靠的系统不仅应能够检测分布外（OOD）输入，还应能够通过为潜在误分类样本分配低置信度来预测分布内（ID）的错误。然而，大多数先前的工作将OOD检测和失败预测视为独立的问题，忽略了它们之间的紧密联系。我们主张可靠性需要联合评估这些方面。为此，我们提出了一种统一的评估框架，将OOD检测和失败预测整合在一起，通过我们新的指标DS-F1和DS-AURC进行量化，其中DS表示双评分函数。在OpenOOD基准测试中，双评分函数产生的分类器比传统单评分方法具有显著更高的可靠性。我们的分析进一步揭示，基于OOD的方法在简单或远OOD位移情况下提供显著增益，但在更具有挑战性的近OOD条件下仅提供微小收益。除了评估之外，我们还扩展了可靠的分类器SURE并引入SURE+，一种显著提高不同场景可靠性的新方法。共同，我们的框架、指标和方法建立了一个新的信任分类基准，并为在现实世界中部署稳健模型提供了实用指导。源代码在https://github.com/Intellindust-AI-Lab/SUREPlus公开。
英文摘要：
Building reliable classifiers is a fundamental challenge for deploying machine learning in real-world applications. A reliable system should not only detect out-of-distribution (OOD) inputs but also anticipate in-distribution (ID) errors by assigning low confidence to potentially misclassified samples. Yet, most prior work treats OOD detection and failure prediction as separated problems, overlooking their closed connection. We argue that reliability requires evaluating them jointly. To this end, we propose a unified evaluation framework that integrates OOD detection and failure prediction, quantified by our new metrics DS-F1 and DS-AURC, where DS denotes double scoring functions. Experiments on the OpenOOD benchmark show that double scoring functions yield classifiers that are substantially more reliable than traditional single scoring approaches. Our analysis further reveals that OOD-based approaches provide notable gains under simple or far-OOD shifts, but only marginal benefits under more challenging near-OOD conditions. Beyond evaluation, we extend the reliable classifier SURE and introduce SURE+, a new approach that significantly improves reliability across diverse scenarios. Together, our framework, metrics, and method establish a new benchmark for trustworthy classification and offer practical guidance for deploying robust models in real-world settings. The source code is publicly available at https://github.com/Intellindust-AI-Lab/SUREPlus.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
