arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2105.09394cs.LG
收藏

使用特征对比学习平衡鲁棒性和灵敏度

Balancing Robustness and Sensitivity using Feature Contrastive Learning

Google Research(谷歌研究院);University of California, Los Angeles(加利福尼亚大学洛杉矶分校)

Seungyeon Kim, Daniel Glasner, Srikumar Ramalingam, Cho-Jui Hsieh, Kishore Papineni, Sanjiv Kumar

摘要:

人们普遍认为,对超大型网络进行健壮的训练对于它们在现实世界中的成功至关重要。然而,当采取极端措施时,提高鲁棒性的方法可能会损害模型对罕见或代表性不足的模式的敏感性。在本文中,我们讨论了敏感性和鲁棒性之间的权衡自然(非对抗性)扰动引入两个概念:上下文特征效用和上下文特征敏感性。我们提出的功能对比学习(FCL),鼓励模型更敏感的功能,具有较高的上下文效用。实证结果表明,使用FCL训练的模型实现了鲁棒性和灵敏度的更好平衡,从而在视觉和NLP数据集上存在噪声的情况下提高了泛化能力。

英文摘要:

It is generally believed that robust training of extremely large networks is critical to their success in real-world applications. However, when taken to the extreme, methods that promote robustness can hurt the model's sensitivity to rare or underrepresented patterns. In this paper, we discuss this trade-off between sensitivity and robustness to natural (non-adversarial) perturbations by introducing two notions: contextual feature utility and contextual feature sensitivity. We propose Feature Contrastive Learning (FCL) that encourages a model to be more sensitive to the features that have higher contextual utility. Empirical results demonstrate that models trained with FCL achieve a better balance of robustness and sensitivity, leading to improved generalization in the presence of noise on both vision and NLP datasets.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑