arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2406.09745cs.LG
收藏

分布匹配如何助力领域泛化:一项信息论分析

How Does Distribution Matching Help Domain Generalization: An Information-theoretic Analysis

College of Science, Huazhong Agriculture University(华中农业大学理学院);China Telecom Corporation(中国电信集团有限公司)

Yuxin Dong, Tieliang Gong, Hong Chen, Shuangyong Song, Weizhan Zhang, Chen Li

AI总结:

本文从信息论和概率视角分析分布匹配对领域泛化的作用,揭示梯度匹配与表示匹配的互补关系,并提出IDM与PDM方法以同时对齐梯度和表示,从而提升泛化性能。

摘要:

领域泛化旨在跨多个训练领域学习不变性,从而增强对分布外数据的泛化能力。尽管梯度或表示匹配算法已取得显著成功,但这些方法通常缺乏泛化保证或依赖强假设,导致对分布匹配底层机制的理解仍存在空白。在本工作中,我们从一种新的概率视角来形式化领域泛化,在确保鲁棒性的同时避免过于保守的解。通过全面的信息论分析,我们为梯度和表示匹配在促进泛化中的作用提供了关键见解。我们的结果揭示了这两个组成部分之间的互补关系,表明现有仅关注梯度对齐或表示对齐的工作不足以解决领域泛化问题。基于这些理论发现,我们提出了IDM,以同时对齐领域间梯度和表示。结合所提出的用于复杂分布匹配的PDM方法,IDM在各种基线方法上取得了更优越的性能。

英文摘要:

Domain generalization aims to learn invariance across multiple training domains, thereby enhancing generalization against out-of-distribution data. While gradient or representation matching algorithms have achieved remarkable success, these methods generally lack generalization guarantees or depend on strong assumptions, leaving a gap in understanding the underlying mechanism of distribution matching. In this work, we formulate domain generalization from a novel probabilistic perspective, ensuring robustness while avoiding overly conservative solutions. Through comprehensive information-theoretic analysis, we provide key insights into the roles of gradient and representation matching in promoting generalization. Our results reveal the complementary relationship between these two components, indicating that existing works focusing solely on either gradient or representation alignment are insufficient to solve the domain generalization problem. In light of these theoretical findings, we introduce IDM to simultaneously align the inter-domain gradients and representations. Integrated with the proposed PDM method for complex distribution matching, IDM achieves superior performance over various baseline methods.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑