arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1903.06256cs.CVcs.LG
收藏

通过投影表面统计量来学习鲁棒表示

Learning Robust Representations by Projecting Superficial Statistics Out

Carnegie Mellon University(卡内基梅隆大学);Beijing Normal University(北京师范大学)

Haohan Wang, Zexue He, Zachary C. Lipton, Eric P. Xing

摘要:

尽管在i.i.d.上的评估表现令人印象深刻。保持数据,深度神经网络严重依赖于训练数据的表面统计,并且容易在分布偏移下崩溃。例如,图像背景或纹理的细微变化可能会破坏看似强大的分类器。基于以前在领域泛化方面的工作,我们希望产生一个分类器,即使在训练过程中没有域标识符,它也能泛化到以前看不见的域。这种设置具有挑战性,因为模型可能会提取许多分布特定的(表面)信号以及分布不可知的(语义)信号。为了克服这一挑战,我们采用灰度共生矩阵(GLCM)来提取我们的先验知识表明是肤浅的模式:它们对纹理敏感,但无法捕捉图像的完形。然后,我们介绍了两种技术,以提高我们的网络的样本外的性能。第一种方法建立在反向梯度方法的基础上,该方法推动我们的模型学习GLCM表示不可预测的表示。第二种方法是建立在通过将模型的表示投影到与GLCM表示正交的子空间上而引入的独立性上。我们在标准领域泛化数据集上测试了我们的方法,有趣的是,与其他明确需要目标分布样本进行训练的领域泛化方法相比,我们的方法获得了相当或更好的性能。

英文摘要:

Despite impressive performance as evaluated on i.i.d. holdout data, deep neural networks depend heavily on superficial statistics of the training data and are liable to break under distribution shift. For example, subtle changes to the background or texture of an image can break a seemingly powerful classifier. Building on previous work on domain generalization, we hope to produce a classifier that will generalize to previously unseen domains, even when domain identifiers are not available during training. This setting is challenging because the model may extract many distribution-specific (superficial) signals together with distribution-agnostic (semantic) signals. To overcome this challenge, we incorporate the gray-level co-occurrence matrix (GLCM) to extract patterns that our prior knowledge suggests are superficial: they are sensitive to the texture but unable to capture the gestalt of an image. Then we introduce two techniques for improving our networks' out-of-sample performance. The first method is built on the reverse gradient method that pushes our model to learn representations from which the GLCM representation is not predictable. The second method is built on the independence introduced by projecting the model's representation onto the subspace orthogonal to GLCM representation's. We test our method on the battery of standard domain generalization data sets and, interestingly, achieve comparable or better performance as compared to other domain generalization methods that explicitly require samples from the target distribution for training.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑