arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2203.09125cs.CVcs.AIcs.LG
收藏

视觉Transformer对伪相关性具有鲁棒性吗?

Are Vision Transformers Robust to Spurious Correlations?

University of Wisconsin-Madison(威斯康星大学麦迪逊分校)

Soumya Suvra Ghosal, Yifei Ming, Yixuan Li

摘要:

深度神经网络可能容易学习到虚假的相关性,这些相关性在平均水平上成立,但在非典型测试样本中则不成立。随着最近出现的视觉Transformer(ViT)模型,它仍然是未充分探讨如何虚假的相关性表现在这样的架构。在本文中,我们系统地研究了视觉Transformer对三个具有挑战性的基准数据集上的伪相关性的鲁棒性,并将其性能与流行的CNN进行了比较。我们的研究表明,当在足够大的数据集上进行预训练时,ViT模型比CNN对虚假相关性更鲁棒。他们成功的关键是能够从虚假相关不成立的例子中更好地概括。此外,我们进行了广泛的消融和实验,以了解自我注意力机制在虚假相关环境下提供鲁棒性的作用。我们希望我们的工作将激发未来的研究,进一步了解ViT模型的鲁棒性。

英文摘要:

Deep neural networks may be susceptible to learning spurious correlations that hold on average but not in atypical test samples. As with the recent emergence of vision transformer (ViT) models, it remains underexplored how spurious correlations are manifested in such architectures. In this paper, we systematically investigate the robustness of vision transformers to spurious correlations on three challenging benchmark datasets and compare their performance with popular CNNs. Our study reveals that when pre-trained on a sufficiently large dataset, ViT models are more robust to spurious correlations than CNNs. Key to their success is the ability to generalize better from the examples where spurious correlations do not hold. Further, we perform extensive ablations and experiments to understand the role of the self-attention mechanism in providing robustness under spuriously correlated environments. We hope that our work will inspire future research on further understanding the robustness of ViT models.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑