SWAD：寻找平坦极小值的领域泛化
SWAD: Domain Generalization by Seeking Flat Minima
摘要：
领域泛化（DG）方法旨在通过仅使用来自源领域的训练数据来实现对未知目标领域的泛化。虽然已经提出了各种DG方法，最近的一项研究表明，在一个公平的评估协议，称为DomainBed，简单的经验风险最小化（ERM）的方法的作品相当，甚至优于以前的方法。不幸的是，简单地解决一个复杂的，非凸的损失函数的ERM可以很容易地导致次优的推广，寻求尖锐的最小值。在本文中，我们从理论上表明，找到平坦的最小结果在一个较小的领域泛化差距。我们还提出了一个简单而有效的方法，命名为随机加权平均密度（SWAD），找到平坦的最小值。SWAD发现平坦的最小值和遭受过拟合比香草SWA的密集和过拟合意识的随机权重抽样策略。SWAD在五个DG基准测试中表现出最先进的性能，即PACS，VLCS，ECONOMY Home，TerraIncognita和DomainNet，在域外准确性上平均具有+1.6%的一致性和较大的利润率。我们还比较了SWAD与传统的泛化方法，如数据增强和一致性正则化方法，以验证显着的性能改善源于寻求平坦的最小值，而不是从更好的域内泛化。最后但并非最不重要的是，SWAD是很容易适应现有的DG方法无需修改，SWAD和现有的DG方法的组合进一步提高DG性能。源代码可在https://github.com/khanrc/swad上获得。
英文摘要：
Domain generalization (DG) methods aim to achieve generalizability to an unseen target domain by using only training data from the source domains. Although a variety of DG methods have been proposed, a recent study shows that under a fair evaluation protocol, called DomainBed, the simple empirical risk minimization (ERM) approach works comparable to or even outperforms previous methods. Unfortunately, simply solving ERM on a complex, non-convex loss function can easily lead to sub-optimal generalizability by seeking sharp minima. In this paper, we theoretically show that finding flat minima results in a smaller domain generalization gap. We also propose a simple yet effective method, named Stochastic Weight Averaging Densely (SWAD), to find flat minima. SWAD finds flatter minima and suffers less from overfitting than does the vanilla SWA by a dense and overfit-aware stochastic weight sampling strategy. SWAD shows state-of-the-art performances on five DG benchmarks, namely PACS, VLCS, OfficeHome, TerraIncognita, and DomainNet, with consistent and large margins of +1.6% averagely on out-of-domain accuracy. We also compare SWAD with conventional generalization methods, such as data augmentation and consistency regularization methods, to verify that the remarkable performance improvements are originated from by seeking flat minima, not from better in-domain generalizability. Last but not least, SWAD is readily adaptable to existing DG methods without modification; the combination of SWAD and an existing DG method further improves DG performances. Source code is available at https://github.com/khanrc/swad.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
