arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.14966stat.MLcs.LGstat.ME
收藏

平滑度自适应假设迁移学习

Smoothness Adaptive Hypothesis Transfer Learning

The Pennsylvania State University(宾夕法尼亚州立大学);Amazon Science(亚马逊科学)

Haotian Lin, Matthew Reimherr

AI总结:

本文提出平滑度自适应迁移学习(SATL),在两阶段核岭回归中采用高斯核自适应未知平滑度,证明其达到极小极大最优收敛速率并优于非迁移学习。

摘要:

许多现有的两阶段基于核的假设迁移学习算法在各阶段采用相同的核正则化,并依赖已知的函数平滑度来获得最优性。因此,它们无法适应实践中目标/源及其偏移函数之间变化且未知的平滑度。在本文中,我们通过提出平滑度自适应迁移学习(SATL)来解决这些问题,SATL是一种基于两阶段核岭回归(KRR)的算法。我们首先证明,在仅目标KRR学习中使用错误指定的固定带宽高斯核可以达到极小极大最优性,并推导出针对未知Sobolev平滑度的自适应过程。利用这些结果,SATL在两个阶段均采用高斯核,使得估计器能够适应目标/源及其偏移函数的未知平滑度。我们推导了该学习问题在超额风险下的极小极大下界,并证明SATL享有匹配的上界,仅相差一个对数因子。极小极大收敛速率揭示了影响迁移动态的因素,并证明了SATL相比非迁移学习设置的优越性。虽然我们的主要目标是理论分析,但我们也进行了若干实验以验证我们的结果。

英文摘要:

Many existing two-phase kernel-based hypothesis transfer learning algorithms employ the same kernel regularization across phases and rely on the known smoothness of functions to obtain optimality. Therefore, they fail to adapt to the varying and unknown smoothness between the target/source and their offset in practice. In this paper, we address these problems by proposing Smoothness Adaptive Transfer Learning (SATL), a two-phase kernel ridge regression(KRR)-based algorithm. We first prove that employing the misspecified fixed bandwidth Gaussian kernel in target-only KRR learning can achieve minimax optimality and derive an adaptive procedure to the unknown Sobolev smoothness. Leveraging these results, SATL employs Gaussian kernels in both phases so that the estimators can adapt to the unknown smoothness of the target/source and their offset function. We derive the minimax lower bound of the learning problem in excess risk and show that SATL enjoys a matching upper bound up to a logarithmic factor. The minimax convergence rate sheds light on the factors influencing transfer dynamics and demonstrates the superiority of SATL compared to non-transfer learning settings. While our main objective is a theoretical analysis, we also conduct several experiments to confirm our results.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑