arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2406.16608cs.LGcs.CV
收藏

当不变表示学习遇到标签偏移:不足性与理论洞见

When Invariant Representation Learning Meets Label Shift: Insufficiency and Theoretical Insights

School of Mathematics, Sun Yat-Sen University(中山大学数学学院)

You-Wei Luo, Chuan-Xian Ren

AI总结:

本文针对数据集偏移场景,探究现有不变表示学习的不足,从理论上证明广义标签偏移(GLS)校正对泛化的充要性,提出基于核嵌入的校正算法KECA,实验验证其有效性与优越性。

摘要:

作为面向环境变化的真实世界学习场景的关键步骤,数据集偏移理论与不变表示学习算法已得到广泛研究,用于放宽经典学习设定中的同分布假设。在关于分布偏移本质的不同假设中,广义标签偏移(GLS)是最新提出的假设,在应对偏移中的复杂因素方面展现出巨大潜力。本文旨在探究当前数据集偏移理论与算法的局限性,并通过对GLS的全面理解进一步提供新的洞见。理论层面,本文推导了两个具有指导意义的泛化界,且从贝叶斯视角证明了GLS学习器与最优目标模型足够接近。主要结果表明了不变表示学习的不足性,并证明了GLS校正对于泛化的充分性与必要性,为探索数据集偏移下的可泛化模型提供了理论支撑与创新。方法层面,本文对现有偏移校正框架给出了统一视角,并提出了一种基于核嵌入的校正算法(KECA),以最小化泛化误差并实现成功的知识迁移。理论结果与大量实验评估均证明了GLS校正在解决数据集偏移问题上的充分性与必要性,以及所提算法的优越性。

英文摘要:

As a crucial step toward real-world learning scenarios with changing environments, dataset shift theory and invariant representation learning algorithm have been extensively studied to relax the identical distribution assumption in classical learning setting. Among the different assumptions on the essential of shifting distributions, generalized label shift (GLS) is the latest developed one which shows great potential to deal with the complex factors within the shift. In this paper, we aim to explore the limitations of current dataset shift theory and algorithm, and further provide new insights by presenting a comprehensive understanding of GLS. From theoretical aspect, two informative generalization bounds are derived, and the GLS learner is proved to be sufficiently close to optimal target model from the Bayesian perspective. The main results show the insufficiency of invariant representation learning, and prove the sufficiency and necessity of GLS correction for generalization, which provide theoretical supports and innovations for exploring generalizable model under dataset shift. From methodological aspect, we provide a unified view of existing shift correction frameworks, and propose a kernel embedding-based correction algorithm (KECA) to minimize the generalization error and achieve successful knowledge transfer. Both theoretical results and extensive experiment evaluations demonstrate the sufficiency and necessity of GLS correction for addressing dataset shift and the superiority of proposed algorithm.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑