arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2209.00652cs.LG
收藏

面向领域泛化的优化与模型选择:一种Mixup引导的解决方案

Towards Optimization and Model Selection for Domain Generalization: A Mixup-guided Solution

Tsinghua University(清华大学);Microsoft Research Asia(微软亚洲研究院);National Engineering Research Center for Software Engineering, Peking University(北京大学软件工程国家工程研究中心)

Wang Lu, Jindong Wang, Yidong Wang, Xing Xie

AI总结:

针对领域泛化中长期被忽视的正则化目标优化与模型选择问题,提出利用改进的Mixup生成分布外数据集引导Pareto优化,并构建更接近目标分布的验证集进行模型选择,显著提升了现有算法性能并达到新SOTA。

摘要:

训练数据和测试数据之间的分布偏移通常会破坏模型的性能。近年来,许多研究关注存在分布偏移且目标数据不可见的领域泛化(DG)。尽管在算法设计上取得了进展,但两个基础因素长期被忽视:1)基于正则化目标的优化,以及2)由于无法利用关于目标领域的知识而面临的DG模型选择。本文提出了针对DG的Mixup引导的优化和选择技术。在优化方面,我们利用一种改进的Mixup生成一个分布外数据集,以引导偏好方向并使用Pareto优化进行优化。在模型选择方面,我们生成一个距离目标分布更近的验证数据集,从而能更好地表示目标数据。我们还提出了提案背后的一些理论见解。综合实验表明,我们的模型优化和选择技术能大幅提升现有领域泛化算法的性能,甚至取得了新的最先进结果。

英文摘要:

The distribution shifts between training and test data typically undermine the performance of models. In recent years, lots of work pays attention to domain generalization (DG) where distribution shifts exist, and target data are unseen. Despite the progress in algorithm design, two foundational factors have long been ignored: 1) the optimization for regularization-based objectives, and 2) the model selection for DG since no knowledge about the target domain can be utilized. In this paper, we propose Mixup guided optimization and selection techniques for DG. For optimization, we utilize an adapted Mixup to generate an out-of-distribution dataset that can guide the preference direction and optimize with Pareto optimization. For model selection, we generate a validation dataset with a closer distance to the target distribution, and thereby it can better represent the target data. We also present some theoretical insights behind our proposals. Comprehensive experiments demonstrate that our model optimization and selection techniques can largely improve the performance of existing domain generalization algorithms and even achieve new state-of-the-art results.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑