arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2412.09795cs.LGcs.CV
收藏

是模型还是指标——关于深度学习模型的鲁棒性度量

Is it the model or the metric -- On robustness measures of deeplearning models

Technical University of Denmark(丹麦技术大学);King’s College London(伦敦国王学院)

Zhijin Lyu, Yutong Jin, Sneha Das

AI总结:

本研究针对深度学习模型鲁棒性评估难题,在深度伪造检测场景下提出鲁棒比率(RR)作为鲁棒准确率(RA)的补充指标,证明RA相近的模型在不同扰动水平下RR存在差异,为模型鲁棒性评估提供更全面的度量方式。

摘要:

在自动化决策系统中,确定深度学习模型的鲁棒性是一项已存在且持续存在的挑战。随着实现先进深度学习(DL)的技术出现并取得成功,这些模型被广泛应用于各类场景,包括医疗保健、教育、边境管控等高风险领域。因此,了解这些模型的局限性并预测其失效区域,从而为其成功且安全的部署设置必要的防护措施至关重要。在本研究中,我们重新探讨鲁棒性问题,重点在深度伪造检测(deepfake detection)语境下研究鲁棒准确率(robust accuracy, RA)的充分性。我们提出鲁棒比率(robust ratio, RR)作为补充指标,可量化输入扰动下归一化结果或概率结果的变化。我们对RA和RR进行了对比,证明尽管模型间的RA相近,但在不同的容忍(扰动)水平下,模型表现出的RR存在差异。

英文摘要:

Determining the robustness of deep learning models is an established and ongoing challenge within automated decision-making systems. With the advent and success of techniques that enable advanced deep learning (DL), these models are being used in widespread applications, including high-stake ones like healthcare, education, border-control. Therefore, it is critical to understand the limitations of these models and predict their regions of failures, in order to create the necessary guardrails for their successful and safe deployment. In this work, we revisit robustness, specifically investigating the sufficiency of robust accuracy (RA), within the context of deepfake detection. We present robust ratio (RR) as a complementary metric, that can quantify the changes to the normalized or probability outcomes under input perturbation. We present a comparison of RA and RR and demonstrate that despite similar RA between models, the models show varying RR under different tolerance (perturbation) levels.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑