arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2410.23461cs.LG
收藏

面向分布外泛化的变换不变学习与理论保证

Transformation-Invariant Learning and Theoretical Guarantees for OOD Generalization

Yale University(耶鲁大学);Harvard University(哈佛大学);EPFL(洛桑联邦理工学院);Apple(苹果公司)

Omar Montasser, Han Shao, Emmanuel Abbe

AI总结:

本文针对训练与测试分布可通过数据变换关联的分布偏移场景,开展理论研究,建立学习规则及到ERM的算法约简,给出基于复合类VC维的样本复杂度上界,并提出该问题的博弈论解释。

摘要:

训练与测试分布一致场景下的学习已在实践和理论层面得到广泛研究,但分布偏移下的统计学习仍有诸多问题有待厘清。本文聚焦一类分布偏移场景:训练与测试分布可通过若干(数据)变换映射族建立关联。本文率先对该框架展开理论研究,考察目标变换族已知和未知两种学习场景。本文建立了相应的学习规则,以及到经验风险最小化(Empirical Risk Minimization, ERM)的算法约简,并给出了学习保证。本文基于“预测器与变换复合后的函数类”的VC维,得到了样本复杂度上界;研究表明,多数情况下该复合类的VC维仅略大于预测器类本身的VC维。本文强调,所推导的学习规则为分布偏移问题提供了博弈论视角:学习者寻找预测器以最小化最坏情况损失,而敌手寻找变换映射以最大化该损失。

英文摘要:

Learning with identical train and test distributions has been extensively investigated both practically and theoretically. Much remains to be understood, however, in statistical learning under distribution shifts. This paper focuses on a distribution shift setting where train and test distributions can be related by classes of (data) transformation maps. We initiate a theoretical study for this framework, investigating learning scenarios where the target class of transformations is either known or unknown. We establish learning rules and algorithmic reductions to Empirical Risk Minimization (ERM), accompanied with learning guarantees. We obtain upper bounds on the sample complexity in terms of the VC dimension of the class composing predictors with transformations, which we show in many cases is not much larger than the VC dimension of the class of predictors. We highlight that the learning rules we derive offer a game-theoretic viewpoint on distribution shift: a learner searching for predictors and an adversary searching for transformation maps to respectively minimize and maximize the worst-case loss.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑