arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2201.00057cs.LGcs.AIcs.ITmath.ITstat.ML
收藏

协变量偏移的最优表示

Optimal Representations for Covariate Shift

Vector Institute(向量研究所);University of Toronto(多伦多大学)

Yangjun Ruan, Yann Dubois, Chris J. Maddison

摘要:

机器学习系统经常在训练和测试之间经历分布偏移。在本文中,我们引入了一个简单的变分目标,其最优解正好是所有表示的集合,在这些表示上,风险最小化器保证对任何保持贝叶斯预测的分布偏移都是鲁棒的,例如,协变量偏移。我们的目标有两个组成部分。首先,表示必须保持对任务的区分性,即,某些预测器必须能够同时最小化源和目标风险。其次,表示的边际支持在源和目标之间需要相同。我们通过设计仅使用未标记数据和增强来训练鲁棒表示的自监督目标来实现这一点。我们的目标是深入了解CLIP的鲁棒性,并进一步改进CLIP的表示,以在DomainBed上实现SOTA结果。

英文摘要:

Machine learning systems often experience a distribution shift between training and testing. In this paper, we introduce a simple variational objective whose optima are exactly the set of all representations on which risk minimizers are guaranteed to be robust to any distribution shift that preserves the Bayes predictor, e.g., covariate shifts. Our objective has two components. First, a representation must remain discriminative for the task, i.e., some predictor must be able to simultaneously minimize the source and target risk. Second, the representation's marginal support needs to be the same across source and target. We make this practical by designing self-supervised objectives that only use unlabelled data and augmentations to train robust representations. Our objectives give insights into the robustness of CLIP, and further improve CLIP's representations to achieve SOTA results on DomainBed.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑