arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2403.09869stat.MLcs.AIcs.LGstat.ME
收藏

注意GAP:利用组感知先验提高对子群体偏移的鲁棒性

Mind the GAP: Improving Robustness to Subpopulation Shifts with Group-Aware Priors

New York University(纽约大学);Meta FAIR

Tim G. J. Rudner, Ya Shi Zhang, Andrew Gordon Wilson, Julia Kempe

AI总结:

本文提出组感知先验(GAP)方法,通过利用少量带组信息的数据训练模型,显著提升模型在子群体偏移下的鲁棒性,并达到最先进性能。

摘要:

机器学习模型在数据分布发生子群体偏移时往往表现不佳。开发能够使机器学习模型更好地泛化到此类偏移的方法,对于在现实环境中安全部署至关重要。在本文中,我们开发了一族基于神经网络参数的组感知先验(GAP)分布,这些分布明确偏好能够在子群体偏移下良好泛化的模型。我们设计了一种简单的组感知先验,它仅需要访问一小部分带有组信息的数据,并证明使用该先验进行训练能够达到最先进的性能——即使仅对先前训练的非鲁棒模型的最后一层进行重新训练也是如此。组感知先验在概念上简单,与属性伪标签和数据重加权等现有方法互补,并为利用贝叶斯推断实现子群体偏移鲁棒性开辟了有前景的新途径。

英文摘要:

Machine learning models often perform poorly under subpopulation shifts in the data distribution. Developing methods that allow machine learning models to better generalize to such shifts is crucial for safe deployment in real-world settings. In this paper, we develop a family of group-aware prior (GAP) distributions over neural network parameters that explicitly favor models that generalize well under subpopulation shifts. We design a simple group-aware prior that only requires access to a small set of data with group information and demonstrate that training with this prior yields state-of-the-art performance -- even when only retraining the final layer of a previously trained non-robust model. Group aware-priors are conceptually simple, complementary to existing approaches, such as attribute pseudo labeling and data reweighting, and open up promising new avenues for harnessing Bayesian inference to enable robustness to subpopulation shifts.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑