This paper investigates the effectiveness of transfer learning based on Mallows' Cp. We propose a procedure that combines transfer learning with Mallows' Cp (TLCp) and prove that it outperforms the conventional Mallows' Cp criterion in terms of accuracy and stability. Our theoretical results indicate that, for any sample size in the target domain, the proposed TLCp estimator performs better than the Cp estimator by the mean squared error (MSE) metric in the case of orthogonal predictors, provided that i) the dissimilarity between the tasks from source domain and target domain is small, and ii) the procedure parameters (complexity penalties) are tuned according to certain explicit rules. Moreover, we show that our transfer learning framework can be extended to other feature selection criteria, such as the Bayesian information criterion. By analyzing the solution of the orthogonalized Cp, we identify an estimator that asymptotically approximates the solution of the Cp criterion in the case of non-orthogonal predictors. Similar results are obtained for the non-orthogonal TLCp. Finally, simulation studies and applications with real data demonstrate the usefulness of the TLCp scheme.
Bridging Information Criteria and Parameter Shrinkage for Model Selection
Max Planck Institute for Intelligent Systems(马克斯·普朗克智能系统研究所);Hong Kong Baptist University(香港浸会大学);Chinese University of Hong Kong(香港中文大学);University of Helsinki(赫尔辛基大学)
Information-theoretic Online Memory Selection for Continual Learning
DeepMind(深度思维);University of Toronto(多伦多大学);Vector Institute(向量研究所);Google Brain(谷歌大脑);Baidu Apollo(百度阿波罗)
无任务持续学习中的一个挑战性问题是从数据流中在线选择代表性重放记忆。在这项工作中,我们从信息论的角度研究了在线记忆选择问题。为了收集最多的信息,我们提出了\textit{surprise}和\textit{learnability}标准来挑选信息点并避免离群值。我们提出了一个贝叶斯模型来计算的标准,有效地利用秩一矩阵结构。我们证明,这些标准鼓励选择信息点在线记忆选择的贪心算法。此外,通过识别\textit{the timing to update the memory}的重要性,我们介绍了随机信息理论水库采样器(InfoRS),它进行高信息的选择点之间的采样。与水库采样相比,InfoRS显示出对数据不平衡的更好的鲁棒性。最后,通过对持续学习基准的实证分析,证明了该方法的有效性。
A Comparative Study of Model Selection Criteria for Symbolic Regression
Aarhus University(奥胡斯大学);Heuristic and Evolutionary Algorithms Laboratory (HEAL)(启发式和进化算法实验室);Federal University of ABC(巴西联邦大学ABC分校);University of Modena and Reggio Emilia(摩德纳和雷吉奥艾米利亚大学);University of Modena(摩德纳大学);Reggio Emilia Department of Physics, Informatics(雷吉奥艾米利亚物理与信息学系)
Utilizing stability criteria in choosing feature selection methods yields reproducible results in microbiome data
University of California San Diego(加州大学圣地亚哥分校);IBM T. J. Watson Research Center(IBM托马斯·J·沃森研究中心);IBM Research UK(IBM英国研究院);IBM Research-Almaden(IBM阿尔马登研究院)
L1 logistic regression as a feature selection step for training stable classification trees for the prediction of severity criteria in imported malaria
École Centrale Paris(巴黎中央理工学院);Institut HyperCube(超立方体研究所);Hopital Bichat-Claude Bernard(比沙-克洛德·贝尔纳医院)
Estimation and Model Selection for Model-Based Clustering with the Conditional Classification Likelihood
Biernacki et al.(2000)在基于模型的聚类框架中提出了综合完全似然(ICL)标准,以选择相关数量的类,并已被统计学家用于各种应用领域。提出了一个理论研究这个标准。引入了一个与聚类目标相关的对比:条件分类似然。这将产生一个估计量和一个模型选择标准类。这些新的程序的性质进行了研究和ICL被证明是这些标准之一的近似。我们反对这些结果,以目前领先的观点,关于ICL,这将是不一致的。此外,这些结果提供了深入了解ICL的类概念和饲料的聚类类概念的反射。惩罚的最小对比度准则和混合模型的一般结果,这是有趣的,在自己的权利。