arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2107.02847stat.MLcs.LG
收藏

基于信息准则的特征选择中的迁移学习

Transfer Learning in Information Criteria-based Feature Selection

Zhejiang University(浙江大学);Georgia Institute of Technology(佐治亚理工学院)

Shaohan Chen, Nikolaos V. Sahinidis, Chuanhou Gao

摘要:

本文研究了基于Mallows' Cp的迁移学习的有效性。我们提出了一种结合迁移学习和Malows ' Cp(TLCp)的方法,并证明了它在精度和稳定性方面优于传统的Malows' Cp准则。我们的理论结果表明,对于任何样本量的目标域,建议TLCp估计性能优于Cp估计的均方误差(MSE)度量的正交预测的情况下,提供i)之间的差异从源域和目标域的任务是小的,和ii)的过程参数(复杂性惩罚)根据某些明确的规则进行调整。此外,我们证明了我们的迁移学习框架可以扩展到其他特征选择标准,如贝叶斯信息标准。通过分析的解决方案的正交Cp,我们确定了一个估计渐近逼近的解决方案的Cp准则的情况下,非正交的预测。对于非正交的TLCp也得到了类似的结果。最后,模拟研究和实际数据的应用表明TLCp计划的有效性。

英文摘要:

This paper investigates the effectiveness of transfer learning based on Mallows' Cp. We propose a procedure that combines transfer learning with Mallows' Cp (TLCp) and prove that it outperforms the conventional Mallows' Cp criterion in terms of accuracy and stability. Our theoretical results indicate that, for any sample size in the target domain, the proposed TLCp estimator performs better than the Cp estimator by the mean squared error (MSE) metric in the case of orthogonal predictors, provided that i) the dissimilarity between the tasks from source domain and target domain is small, and ii) the procedure parameters (complexity penalties) are tuned according to certain explicit rules. Moreover, we show that our transfer learning framework can be extended to other feature selection criteria, such as the Bayesian information criterion. By analyzing the solution of the orthogonalized Cp, we identify an estimator that asymptotically approximates the solution of the Cp criterion in the case of non-orthogonal predictors. Similar results are obtained for the non-orthogonal TLCp. Finally, simulation studies and applications with real data demonstrate the usefulness of the TLCp scheme.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑