arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2412.04245cs.CV
收藏

鲁棒分类的有趣性质

Intriguing Properties of Robust Classification

Institute of Science and Technology Austria (ISTA)(奥地利科学与技术研究所)

Bernd Prach, Christoph H. Lampert

AI总结:

本文通过理论分析证明鲁棒分类可能需要指数级训练数据,并在CIFAR-10上验证数据量是决定鲁棒性能的主要因素,揭示了鲁棒与非鲁棒泛化在数据利用上的本质差异。

摘要:

尽管自社区在10年前了解到对抗样本以来进行了大量研究,我们仍然不知道如何训练出高精度且能保证对其输入的小扰动具有鲁棒性的分类器。以往的工作通常认为,这可能是因为不存在同时具备鲁棒性和准确性的分类器。然而,在计算机视觉中,这一假设与现实不符,因为人类在大多数感兴趣的任务上通常既准确又鲁棒。我们提供了一种替代解释,并表明在某些设定下,鲁棒泛化只有在数据量不切实际地巨大时才可能实现。具体而言,我们发现了一种设定:其中鲁棒分类器是存在的,学习一个准确的分类器很容易,但学习一个鲁棒分类器却需要指数级的数据量。基于这一理论结果,我们评估了训练数据量对CIFAR-10等数据集的影响。我们的发现表明,训练数据量是决定鲁棒性能的主要因素。此外,我们表明数据中存在低幅值方向,这些方向对非鲁棒泛化有用,但对鲁棒分类器不可用。我们在 https://github.com/berndprach/IntriguingProperties 提供了代码。

英文摘要:

Despite extensive research since the community learned about adversarial examples 10 years ago, we still do not know how to train high-accuracy classifiers that are guaranteed to be robust to small perturbations of their inputs. Previous works often argued that this might be because no classifier exists that is robust and accurate at the same time. However, in computer vision this assumption does not match reality where humans are usually accurate and robust on most tasks of interest. We offer an alternative explanation and show that in certain settings robust generalization is only possible with unrealistically large amounts of data. Specifically, we find a setting where a robust classifier exists, it is easy to learn an accurate classifier, yet it requires an exponential amount of data to learn a robust classifier. Based on this theoretical result, we evaluate the influence of the amount of training data on datasets such as CIFAR-10. Our findings indicate that the amount of training data is the main factor determining the robust performance. Furthermore we show that there are low magnitude directions in the data which are useful for non-robust generalization but are not available for robust classifiers. We provide code at https://github.com/berndprach/IntriguingProperties.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑