arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2006.13228stat.MLcs.LG
收藏

一类无实现代价的迁移学习回归算法

A General Class of Transfer Learning Regression without Implementation Cost

The Graduate University for Advanced Studies (SOKENDAI)(高级研究大学(综合研究大学院大学));University of Bristol(布里斯托大学);The Institute of Statistical Mathematics(统计数学研究所);National Institute for Materials Science(国立材料科学研究所)

Shunya Minami, Song Liu, Stephen Wu, Kenji Fukumizu, Ryo Yoshida

摘要:

我们提出了一个新的框架,统一和扩展了现有的迁移学习(TL)回归方法。为了将预训练的源模型与目标任务上的模型连接起来,我们引入了密度比重新加权函数,该函数通过具有特定先验分布的贝叶斯框架进行估计。通过改变两个内在超参数和密度比模型的选择,该方法可以集成三种流行的TL方法:基于跨域相似性正则化的TL,使用密度比估计的概率TL,以及预训练神经网络的微调。此外,所提出的方法可以受益于其简单的实现,而无需任何额外的成本;回归模型可以使用现成的库进行监督学习,其中原始输出变量简单地转换为新的输出变量。我们证明了它的简单性,通用性和适用性,使用各种真实的数据应用程序。

英文摘要:

We propose a novel framework that unifies and extends existing methods of transfer learning (TL) for regression. To bridge a pretrained source model to the model on a target task, we introduce a density-ratio reweighting function, which is estimated through the Bayesian framework with a specific prior distribution. By changing two intrinsic hyperparameters and the choice of the density-ratio model, the proposed method can integrate three popular methods of TL: TL based on cross-domain similarity regularization, a probabilistic TL using the density-ratio estimation, and fine-tuning of pretrained neural networks. Moreover, the proposed method can benefit from its simple implementation without any additional cost; the regression model can be fully trained using off-the-shelf libraries for supervised learning in which the original output variable is simply transformed to a new output variable. We demonstrate its simplicity, generality, and applicability using various real data applications.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑