发表机构
School of Computer Science; University College Dublin（计算机学院; 都柏林大学学院）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本文提出RAD框架，通过模型与特征空间一致性量化预测歧义，经多数据集评估后，将其用于对高歧义样本弃权预测，性能与现有方法相当。
AI 中文摘要
机器学习模型应具备鲁棒性，即在允许的变化下保持预测一致性。当模型被功能等效的模型替换，或输入受到微小、可接受的扰动时，其预测应保持不变；若此类改变显著影响预测，则该预测相对于模型是“歧义的”。在高风险决策场景中，模型应弃权（不执行）此类歧义预测，或标记其供人工检查。然而，模型部署后，此类歧义难以识别。本文提出鲁棒歧义检测（RAD）框架，通过两个互补指标量化预测歧义：模型空间一致性与特征空间一致性。这两个分数（RAD分数对）通过RAD图可视化，可解释地表征歧义来源及用户可采取的应对措施。RAD在具有系统控制重叠的合成数据集，以及若干无法直接检查歧义水平的真实世界数据集上进行了评估。最后，本文展示了RAD的下游应用：按样本的RAD帕累托秩排序，对最具歧义的样本弃权预测，取得了与现有基于拒绝的方法相当的性能。
英文摘要
Machine learning models should be robust, in the sense of remaining predictively consistent under permissible variations. A model's predictions should ideally remain unchanged when it is replaced by a functionally equivalent one, or when its inputs are subject to minor, admissible perturbations. If such changes alter a prediction significantly, then the prediction is "ambiguous" with respect to the model. Models should abstain from making such ambiguous predictions and/or should flag them for human inspection, especially in high-stakes decision-making scenarios. However, in practice, such ambiguity is not easy to identify once a model is deployed. Here, the Robust Ambiguity Detection (RAD) framework is advanced for quantifying predictive ambiguity using two complementary metrics: Model-Space Consistency and Feature-Space Consistency. These two scores, the RAD Score-Pair, visualised through the RAD Plot, provide an interpretable characterisation of the sources of ambiguity and the actions a user may consider in response. RAD is evaluated on synthetic datasets with systematically controlled overlap, as well as several real-world datasets where the level of ambiguity cannot be directly inspected. Finally, we demonstrate a downstream application of RAD where samples are ranked by their RAD Pareto-Rank and the most ambiguous are abstained from prediction, achieving performance comparable to existing rejection-based approaches.
