arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2202.05189cs.LGstat.ML
收藏

理解神经网络中罕见的伪相关性

Understanding Rare Spurious Correlations in Neural Networks

DeepMind(深度思维);University of California San Diego(加州大学圣地亚哥分校);Harvard University(哈佛大学)

Yao-Yuan Yang, Chi-Ning Chou, Kamalika Chaudhuri

摘要:

已知神经网络使用诸如背景信息的伪相关性来进行分类。虽然之前的工作已经研究了训练数据中广泛存在的虚假相关性,但在这项工作中,我们研究了神经网络对罕见的虚假相关性的敏感程度,这可能更难检测和纠正,并可能导致隐私泄露。我们将与固定类相关的虚假模式引入到一些训练样本中,发现网络只需要少数这样的样本就可以学习相关性。此外,这些罕见的虚假相关性也会影响准确性和隐私。我们从经验和理论上分析了罕见的虚假相关性所涉及的不同因素,并提出了相应的缓解方法。具体来说,我们观察到,$\ell_2$正则化和添加高斯噪声的输入可以减少不良影响。代码可在https://github.com/yangarbiter/rare-spurious-correlation获得。

英文摘要:

Neural networks are known to use spurious correlations such as background information for classification. While prior work has looked at spurious correlations that are widespread in the training data, in this work, we investigate how sensitive neural networks are to rare spurious correlations, which may be harder to detect and correct, and may lead to privacy leaks. We introduce spurious patterns correlated with a fixed class to a few training examples and find that it takes only a handful of such examples for the network to learn the correlation. Furthermore, these rare spurious correlations also impact accuracy and privacy. We empirically and theoretically analyze different factors involved in rare spurious correlations and propose mitigation methods accordingly. Specifically, we observe that $\ell_2$ regularization and adding Gaussian noise to inputs can reduce the undesirable effects. Code available at https://github.com/yangarbiter/rare-spurious-correlation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑