arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.02719cs.LG
收藏

多模态临床状况分类中的校准与选择性预测的实证分析

An Empirical Analysis of Calibration and Selective Prediction in Multimodal Clinical Condition Classification

New York University(纽约大学);University of Toronto(多伦多大学)

L. Julián Lechuga López, Farah E. Shamout, Tim G. J. Rudner

AI总结:

本研究通过实证分析多标签临床状况分类中基于不确定性的选择性预测,发现其性能可能严重下降,原因是类别依赖的误校准,尤其对少数类别,并强调需要校准感知评估来确保临床AI的安全性。

摘要:

随着人工智能系统向临床部署迈进,确保可靠的预测行为对于安全关键的决策任务至关重要。一种提议的安全保障是选择性预测,即模型可以将不确定的预测交由人类专家审查。在这项工作中,我们使用多模态ICU数据,实证评估了基于不确定性的选择性预测在多标签临床状况分类中的可靠性。在一系列最先进的单模态和多模态模型中,我们发现尽管标准评估指标表现强劲,但选择性预测可能会大幅降低性能。这种失败是由严重的类别依赖的误校准驱动的,即模型对正确预测赋予高不确定性,对错误预测赋予低不确定性,尤其是对于代表性不足的临床状况。我们的结果表明,常用的聚合指标可能掩盖这些效应,限制了它们评估该设置下选择性预测行为的能力。综合来看,我们的发现描述了多模态临床状况分类中选择性预测的任务特定失败模式,并强调了需要校准感知评估来为临床AI提供强有力的安全性和鲁棒性保证。

英文摘要:

As artificial intelligence systems move toward clinical deployment, ensuring reliable prediction behavior is fundamental for safety-critical decision-making tasks. One proposed safeguard is selective prediction, where models can defer uncertain predictions to human experts for review. In this work, we empirically evaluate the reliability of uncertainty-based selective prediction in multilabel clinical condition classification using multimodal ICU data. Across a range of state-of-the-art unimodal and multimodal models, we find that selective prediction can substantially degrade performance despite strong standard evaluation metrics. This failure is driven by severe class-dependent miscalibration, whereby models assign high uncertainty to correct predictions and low uncertainty to incorrect ones, particularly for underrepresented clinical conditions. Our results show that commonly used aggregate metrics can obscure these effects, limiting their ability to assess selective prediction behavior in this setting. Taken together, our findings characterize a task-specific failure mode of selective prediction in multimodal clinical condition classification and highlight the need for calibration-aware evaluation to provide strong guarantees of safety and robustness in clinical AI.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑