高维线性模型的轮廓迁移学习
Profiled Transfer Learning for High Dimensional Linear Model
AI总结:
本文提出了一种基于近似线性假设的高维线性模型迁移学习方法 PTL,通过两步构建估计量并证明其极小极大最优性,模拟与真实数据均验证了方法的有效性。
摘要:
我们在此开发了一种名为 Profiled Transfer Learning (PTL) 的新颖迁移学习方法。该方法基于源参数与目标参数之间的近似线性假设。与文献中通常假设的差值消失假设和低秩假设相比,近似线性假设更具灵活性且限制更少。具体而言,PTL 估计量由两个主要步骤构建。首先,我们将响应变量对转移特征进行回归,得到轮廓响应。随后,我们在目标数据上学习轮廓响应与协变量之间的回归关系。最终的估计量基于近似线性关系进行组装。为了在理论上支持 PTL 估计量,我们推导了非渐近上界和极小极大下界。我们发现,在适当的正则性条件下,PTL 估计量是极小极大最优的。我们展示了广泛的模拟研究,以证明该新方法的有限样本性能。此外,还提供了一个关于句子预测的真实数据示例,结果非常令人鼓舞。
英文摘要:
We develop here a novel transfer learning methodology called Profiled Transfer Learning (PTL). The method is based on the \textit{approximate-linear} assumption between the source and target parameters. Compared with the commonly assumed \textit{vanishing-difference} assumption and \textit{low-rank} assumption in the literature, the \textit{approximate-linear} assumption is more flexible and less stringent. Specifically, the PTL estimator is constructed by two major steps. Firstly, we regress the response on the transferred feature, leading to the profiled responses. Subsequently, we learn the regression relationship between profiled responses and the covariates on the target data. The final estimator is then assembled based on the \textit{approximate-linear} relationship. To theoretically support the PTL estimator, we derive the non-asymptotic upper bound and minimax lower bound. We find that the PTL estimator is minimax optimal under appropriate regularity conditions. Extensive simulation studies are presented to demonstrate the finite sample performance of the new method. A real data example about sentence prediction is also presented with very encouraging results.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。