arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2006.14845stat.MLcs.LG
收藏

基于$\ell_1$正则化的迁移学习

Transfer Learning via $\ell_1$ Regularization

Toshiba Corporation(东芝公司);The Institute of Statistical Mathematics(统计数理研究所)

Masaaki Takada, Hironori Fujisawa

摘要:

机器学习算法通常需要在静止环境下的大量数据。然而,在许多实际应用中,环境是非平稳的。关键问题在于如何在不断变化的环境下有效地调整模型。我们提出了一种通过$\ell_1$正则化将知识从源域转移到目标域的方法。除了普通的$\ell_1$正则化之外,我们还将源参数和目标参数之间的差异的$\ell_1$正则化。因此,我们的方法产生稀疏的估计本身和估计的变化。该方法在平稳环境下具有严格的估计误差界,且在小残差下估计值与源估计值保持不变。此外,估计是一致的基本功能,即使源估计是错误的,由于非平稳性。实验结果表明,该方法有效地平衡了稳定性和塑性。

英文摘要:

Machine learning algorithms typically require abundant data under a stationary environment. However, environments are nonstationary in many real-world applications. Critical issues lie in how to effectively adapt models under an ever-changing environment. We propose a method for transferring knowledge from a source domain to a target domain via $\ell_1$ regularization. We incorporate $\ell_1$ regularization of differences between source parameters and target parameters, in addition to an ordinary $\ell_1$ regularization. Hence, our method yields sparsity for both the estimates themselves and changes of the estimates. The proposed method has a tight estimation error bound under a stationary environment, and the estimate remains unchanged from the source estimate under small residuals. Moreover, the estimate is consistent with the underlying function, even when the source estimate is mistaken due to nonstationarity. Empirical results demonstrate that the proposed method effectively balances stability and plasticity.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑