带概念防护的分类
Classification with Conceptual Safeguards
AI总结:
提出概念防护作为验证层,通过弃权(不执行)不确定预测确保分类准确率,并允许人工确认概念以提高覆盖率,在真实与合成数据集上验证了有效性。
摘要:
我们提出了一种新方法,利用既定概念来提升分类任务的安全性。该方法被称为“概念防护”,它作为验证层,作用于先预测中间概念存在性、再预测目标结果的模型。在此架构下,防护通过弃权(不执行)不确定的预测,确保模型达到最低准确率。与标准的选择性分类器不同,防护提供了一种提高覆盖率的方式:在模型弃权(不执行)的实例上,允许人类确认不确定概念的存在。我们开发了构建防护的方法,以在不牺牲安全性的前提下最大化覆盖率,即传播概念预测不确定性的技术,以及标记关键概念以供人工审查的技术。我们在真实世界和合成数据集上对方法进行了基准测试,表明它能在深度学习任务中提升性能和覆盖率。
英文摘要:
We propose a new approach to promote safety in classification tasks with established concepts. Our approach -- called a conceptual safeguard -- acts as a verification layer for models that predict a target outcome by first predicting the presence of intermediate concepts. Given this architecture, a safeguard ensures that a model meets a minimal level of accuracy by abstaining from uncertain predictions. In contrast to a standard selective classifier, a safeguard provides an avenue to improve coverage by allowing a human to confirm the presence of uncertain concepts on instances on which it abstains. We develop methods to build safeguards that maximize coverage without compromising safety, namely techniques to propagate the uncertainty in concept predictions and to flag salient concepts for human review. We benchmark our approach on a collection of real-world and synthetic datasets, showing that it can improve performance and coverage in deep learning tasks.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。