arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2104.09937cs.LGstat.ML
收藏

基于梯度匹配的领域泛化方法

Gradient Matching for Domain Generalization

University of Oxford(牛津大学);Facebook Reality Labs(Facebook现实实验室);The University of Edinburgh(爱丁堡大学);The Alan Turing Institute(艾伦·图灵研究所);Facebook AI Research(Facebook人工智能研究院)

Yuge Shi, Jeffrey Seely, Philip H. S. Torr, N. Siddharth, Awni Hannun, Nicolas Usunier, Gabriel Synnaeve

摘要:

机器学习系统通常假设训练集和测试集的分布紧密匹配。然而,在现实世界中,这样的系统的一个关键要求是它们能够推广到看不见的领域。在这里,我们提出了一个域间梯度匹配目标,目标领域泛化通过最大化来自不同领域的梯度之间的内积。由于直接优化的梯度内积可以计算禁止-需要计算的二阶导数-我们推导出一个更简单的一阶算法名为鱼,近似其优化。我们在Wilds基准的6个数据集上证明了Fish的有效性,该基准捕获了各种模式的分布偏移。我们的方法在这些数据集上产生了有竞争力的结果,并在其中4个数据集上超过了所有基线。我们对Wilds基准测试(捕获现实世界中的分布偏移)以及DomainBed基准测试(更侧重于合成到真实传输)中的数据集进行了实验。我们的方法在两个基准上都产生了有竞争力的结果,证明了它在广泛的领域泛化任务中的有效性。

英文摘要:

Machine learning systems typically assume that the distributions of training and test sets match closely. However, a critical requirement of such systems in the real world is their ability to generalize to unseen domains. Here, we propose an inter-domain gradient matching objective that targets domain generalization by maximizing the inner product between gradients from different domains. Since direct optimization of the gradient inner product can be computationally prohibitive -- requires computation of second-order derivatives -- we derive a simpler first-order algorithm named Fish that approximates its optimization. We demonstrate the efficacy of Fish on 6 datasets from the Wilds benchmark, which captures distribution shift across a diverse range of modalities. Our method produces competitive results on these datasets and surpasses all baselines on 4 of them. We perform experiments on both the Wilds benchmark, which captures distribution shift in the real world, as well as datasets in DomainBed benchmark that focuses more on synthetic-to-real transfer. Our method produces competitive results on both benchmarks, demonstrating its effectiveness across a wide range of domain generalization tasks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑