arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1702.03994stat.ML
收藏

metboost:分层聚类数据的探索性回归分析

metboost: Exploratory regression analysis with hierarchically clustered data

Patrick J. Miller, Daniel B. McArtor, Gitta H. Lubke

摘要:

随着数据收集量的增加,探索性回归分析变得更加重要,但也更具挑战性。当观测值分层聚类时,问题更具挑战性,因为当模型中不包括非线性效应时,混合效应模型的模型选择可能会产生误导性结果(Bauer和Cai,2009)。一种称为提升决策树的机器学习方法(Friedman,2001)是在真实数据集中进行探索性回归分析的好方法,因为它可以检测具有非线性和交互效应的预测因子,同时还可以解释缺失数据。我们提出了一个扩展,提升决策树称为metboost的层次聚类数据。它的工作原理是将每个树的结构约束为跨组相同,但允许终端节点的方式不同。这允许预测因子和分割点在每个组内产生不同的预测,并近似非线性组特定效应。重要的是,metboost对于数千个观测值和数百个可能包含缺失值的预测值仍然是计算可行的。我们应用该方法预测来自751所学校的15,240名学生的数学表现,这些学生的数据来自2002年教育纵向研究(Ingels等人,2007),允许76个预测因子对每所学校都有独特的影响。当将结果与提升的决策树进行比较时,metboost的预测性能提高了15%。一个大型的模拟研究结果表明,当组规模较小时,与提升决策树相比,metboost具有高达70%的变量选择性能和高达30%的预测性能

英文摘要:

As data collections become larger, exploratory regression analysis becomes more important but more challenging. When observations are hierarchically clustered the problem is even more challenging because model selection with mixed effect models can produce misleading results when nonlinear effects are not included into the model (Bauer and Cai, 2009). A machine learning method called boosted decision trees (Friedman, 2001) is a good approach for exploratory regression analysis in real data sets because it can detect predictors with nonlinear and interaction effects while also accounting for missing data. We propose an extension to boosted decision decision trees called metboost for hierarchically clustered data. It works by constraining the structure of each tree to be the same across groups, but allowing the terminal node means to differ. This allows predictors and split points to lead to different predictions within each group, and approximates nonlinear group specific effects. Importantly, metboost remains computationally feasible for thousands of observations and hundreds of predictors that may contain missing values. We apply the method to predict math performance for 15,240 students from 751 schools in data collected in the Educational Longitudinal Study 2002 (Ingels et al., 2007), allowing 76 predictors to have unique effects for each school. When comparing results to boosted decision trees, metboost has 15% improved prediction performance. Results of a large simulation study show that metboost has up to 70% improved variable selection performance and up to 30% improved prediction performance compared to boosted decision trees when group sizes are small

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑