arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2102.11436stat.MLcs.AIcs.LG
收藏

基于模型的领域泛化

Model-Based Domain Generalization

University of Pennsylvania(宾夕法尼亚大学)

Alexander Robey, George J. Pappas, Hamed Hassani

摘要:

尽管在各种应用中取得了显着的成功,但众所周知,当呈现出分布外的数据时,深度学习可能会灾难性地失败。为了解决这一挑战,我们考虑域泛化问题,其中预测器使用从一系列相关训练域中提取的数据进行训练,然后在一个不同的和看不见的测试域上进行评估。我们表明,在自然的数据生成模型和随之而来的不变性条件下,域泛化问题相当于一个无限维的约束统计学习问题,这个问题形成了我们的方法,我们称之为基于模型的域泛化的基础。由于在深度学习中解决约束优化问题的固有挑战,我们利用非凸对偶理论来开发这个统计问题的无约束松弛,并在对偶间隙上有严格的界限。基于这一理论动机,我们提出了一种新的域泛化算法的收敛保证。在我们的实验中,我们报告了在包括ColoredMNIST、Camelyon 17-WILDS、FMoW-WILDS和PACS在内的几个基准测试上,最先进的领域泛化基线提高了30个百分点。

英文摘要:

Despite remarkable success in a variety of applications, it is well-known that deep learning can fail catastrophically when presented with out-of-distribution data. Toward addressing this challenge, we consider the domain generalization problem, wherein predictors are trained using data drawn from a family of related training domains and then evaluated on a distinct and unseen test domain. We show that under a natural model of data generation and a concomitant invariance condition, the domain generalization problem is equivalent to an infinite-dimensional constrained statistical learning problem; this problem forms the basis of our approach, which we call Model-Based Domain Generalization. Due to the inherent challenges in solving constrained optimization problems in deep learning, we exploit nonconvex duality theory to develop unconstrained relaxations of this statistical problem with tight bounds on the duality gap. Based on this theoretical motivation, we propose a novel domain generalization algorithm with convergence guarantees. In our experiments, we report improvements of up to 30 percentage points over state-of-the-art domain generalization baselines on several benchmarks including ColoredMNIST, Camelyon17-WILDS, FMoW-WILDS, and PACS.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑