arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2109.01377cs.LGcs.ITmath.IT
收藏

(在线)迁移学习的贝叶斯方法:理论与算法

A Bayesian Approach to (Online) Transfer Learning: Theory and Algorithms

University of Melbourne(墨尔本大学)

Xuetong Wu, Jonathan H. Manton, Uwe Aickelin, Jingge Zhu

摘要:

迁移学习是一种机器学习范例,其中来自一个问题的知识被用来解决一个新的但相关的问题。虽然可以想象,来自一个任务的知识可能对解决相关任务有用,但如果执行不当,迁移学习算法可能会损害学习性能,而不是提高它-通常称为负迁移。在本文中,我们研究迁移学习从贝叶斯的角度来看,其中使用的参数统计模型。具体来说,我们研究了三种变体的迁移学习问题,瞬时,在线和时变迁移学习。对于每个问题,我们定义了一个适当的目标函数,并提供精确的表达式或上限的学习性能使用信息理论的数量,允许简单和明确的特征时,样本量变得很大。此外,例子表明,即使是小样本量的推导界限是准确的。所获得的界限提供了宝贵的见解先验知识的影响迁移学习,至少相对于我们的贝叶斯公式的迁移学习问题。特别是,我们正式的负迁移发生的条件下的特征。最后,我们设计了两个(在线)迁移学习算法,适合于实际实现,其中一个不需要参数假设。我们用真实的数据集证明了我们的算法的有效性,主要集中在源数据和目标数据具有很强的相似性时。

英文摘要:

Transfer learning is a machine learning paradigm where knowledge from one problem is utilized to solve a new but related problem. While conceivable that knowledge from one task could be useful for solving a related task, if not executed properly, transfer learning algorithms can impair the learning performance instead of improving it -- commonly known as negative transfer. In this paper, we study transfer learning from a Bayesian perspective, where a parametric statistical model is used. Specifically, we study three variants of transfer learning problems, instantaneous, online, and time-variant transfer learning. For each problem, we define an appropriate objective function, and provide either exact expressions or upper bounds on the learning performance using information-theoretic quantities, which allow simple and explicit characterizations when the sample size becomes large. Furthermore, examples show that the derived bounds are accurate even for small sample sizes. The obtained bounds give valuable insights into the effect of prior knowledge for transfer learning, at least with respect to our Bayesian formulation of the transfer learning problem. In particular, we formally characterize the conditions under which negative transfer occurs. Lastly, we devise two (online) transfer learning algorithms that are amenable to practical implementations, one of which does not require the parametric assumption. We demonstrate the effectiveness of our algorithms with real data sets, focusing primarily on when the source and target data have strong similarities.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑