arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2105.01445cs.LGcs.ITmath.IT
收藏

网络迁移学习:负迁移与先前知识的影响

Online Transfer Learning: Negative Transfer and Effect of Prior Knowledge

University of Melbourne(墨尔本大学)

Xuetong Wu, Jonathan H. Manton, Uwe Aickelin, Jingge Zhu

摘要:

迁移学习是一种机器学习范例,其中来自一个任务的知识被用来解决相关任务中的问题。一方面,可以想象,从一个任务中获得的知识可能对解决相关问题有用。另一方面,人们也认识到,如果不正确执行,迁移学习算法实际上可能会损害学习性能,而不是提高它-通常称为“负迁移”。本文研究了源样本以离线方式给出而目标样本顺序到达的在线迁移学习问题。我们定义了在线迁移学习问题的期望后悔,并使用信息理论量提供了后悔的上界。当样本量变大时,我们也得到了精确的边界表达式。实例表明,即使对于小样本量,推导的界限也是准确的。此外,所获得的界限提供了宝贵的洞察力的影响,先验知识的迁移学习在我们的配方。特别是,我们正式的负迁移发生的条件下的特征。

英文摘要:

Transfer learning is a machine learning paradigm where the knowledge from one task is utilized to resolve the problem in a related task. On the one hand, it is conceivable that knowledge from one task could be useful for solving a related problem. On the other hand, it is also recognized that if not executed properly, transfer learning algorithms could in fact impair the learning performance instead of improving it - commonly known as "negative transfer". In this paper, we study the online transfer learning problems where the source samples are given in an offline way while the target samples arrive sequentially. We define the expected regret of the online transfer learning problem and provide upper bounds on the regret using information-theoretic quantities. We also obtain exact expressions for the bounds when the sample size becomes large. Examples show that the derived bounds are accurate even for small sample sizes. Furthermore, the obtained bounds give valuable insight on the effect of prior knowledge for transfer learning in our formulation. In particular, we formally characterize the conditions under which negative transfer occurs.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑