arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2211.10370cs.CVcs.LG
收藏

基于扩散梦想分布偏移的不变学习

Invariant Learning via Diffusion Dreamed Distribution Shifts

University of Maryland(马里兰大学)

Priyatham Kattakinda, Alexander Levine, Soheil Feizi

摘要:

虽然背景是图像分类的一个重要信号,但当前景和背景之间的虚假相关性在测试时被打破时,过度依赖背景会导致不正确的预测。在这些相关性无偏的数据集上进行训练将导致更强大的模型。在本文中,我们提出了这样一个数据集称为扩散梦想分布偏移(D3S)。D3S由通过StableDiffusion生成的合成图像组成,该合成图像使用文本提示和通过将样本前景图像粘贴到背景模板图像上获得的图像引导。使用这种可扩展的方法,我们从10个不同背景中的所有1000个ImageNet类中生成120K的对象图像。由于扩散模型令人难以置信的照片真实感,我们的图像比以前的合成数据集更接近自然图像。D3S包含一个超过17K图像的验证集,其标签在MTurk研究中经过人工验证。使用验证集,我们评估了几种流行的DNN图像分类器,发现模型的分类性能通常会在我们的背景多样化图像上受到影响。接下来,我们利用D3S中的前景和背景标签来学习前景(背景)表示,该前景(背景)表示通过惩罚前景(背景)特征和背景(前景)标签之间的互信息来对背景(前景)的变化保持不变。在这些特征上训练的线性分类器从前景(背景)预测前景(背景)的准确率高达82.9%(93.8%),而从背景和前景预测这些标签的分类器的准确率分别为2.4%和45.6%。这表明我们的前景和背景特征很好地分离了。我们进一步测试这些表示的有效性,通过训练分类器的任务具有很强的虚假相关性。

英文摘要:

Though the background is an important signal for image classification, over reliance on it can lead to incorrect predictions when spurious correlations between foreground and background are broken at test time. Training on a dataset where these correlations are unbiased would lead to more robust models. In this paper, we propose such a dataset called Diffusion Dreamed Distribution Shifts (D3S). D3S consists of synthetic images generated through StableDiffusion using text prompts and image guides obtained by pasting a sample foreground image onto a background template image. Using this scalable approach we generate 120K images of objects from all 1000 ImageNet classes in 10 diverse backgrounds. Due to the incredible photorealism of the diffusion model, our images are much closer to natural images than previous synthetic datasets. D3S contains a validation set of more than 17K images whose labels are human-verified in an MTurk study. Using the validation set, we evaluate several popular DNN image classifiers and find that the classification performance of models generally suffers on our background diverse images. Next, we leverage the foreground & background labels in D3S to learn a foreground (background) representation that is invariant to changes in background (foreground) by penalizing the mutual information between the foreground (background) features and the background (foreground) labels. Linear classifiers trained on these features to predict foreground (background) from foreground (background) have high accuracies at 82.9% (93.8%), while classifiers that predict these labels from background and foreground have a much lower accuracy of 2.4% and 45.6% respectively. This suggests that our foreground and background features are well disentangled. We further test the efficacy of these representations by training classifiers on a task with strong spurious correlations.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑