你只需要一个好的嵌入提取器来修复虚假的相关性
You Only Need a Good Embeddings Extractor to Fix Spurious Correlations
摘要:
训练数据中的虚假相关性通常会导致鲁棒性问题,因为模型会将其用作捷径。例如,当预测一个对象是否是一头牛时,模型可能会学习依赖于它的绿色背景,所以它在沙质背景上的牛上表现不佳。用于衡量缓解此问题的方法的最新技术水平的标准数据集是水鸟。最好的方法(Group Distributionally Robust Optimization - GroupDRO)目前达到了89%的最差组准确率,而原始图像上从头开始的标准训练只能达到72%。GroupDRO要求以端到端的方式使用子组标签训练模型。在本文中,我们证明了我们可以在不使用训练集中任何子组信息的情况下,通过简单地使用来自大型预训练视觉模型提取器的嵌入并在其上训练线性分类器来实现高达90%的准确率。通过在广泛的预训练模型和预训练数据集上的实验,我们证明了预训练模型的容量和预训练数据集的大小很重要。我们的实验表明,与高容量卷积神经网络相比,高容量视觉Transformer的性能更好,并且更大的预训练数据集导致虚假相关数据集上更好的最差组准确度。
英文摘要:
Spurious correlations in training data often lead to robustness issues since models learn to use them as shortcuts. For example, when predicting whether an object is a cow, a model might learn to rely on its green background, so it would do poorly on a cow on a sandy background. A standard dataset for measuring state-of-the-art on methods mitigating this problem is Waterbirds. The best method (Group Distributionally Robust Optimization - GroupDRO) currently achieves 89\% worst group accuracy and standard training from scratch on raw images only gets 72\%. GroupDRO requires training a model in an end-to-end manner with subgroup labels. In this paper, we show that we can achieve up to 90\% accuracy without using any sub-group information in the training set by simply using embeddings from a large pre-trained vision model extractor and training a linear classifier on top of it. With experiments on a wide range of pre-trained models and pre-training datasets, we show that the capacity of the pre-training model and the size of the pre-training dataset matters. Our experiments reveal that high capacity vision transformers perform better compared to high capacity convolutional neural networks, and larger pre-training dataset leads to better worst-group accuracy on the spurious correlation dataset.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。