MetaBags：用于回归的袋装Meta-决策树
MetaBags: Bagged Meta-Decision Trees for Regression
摘要：
集成是解决实际监督学习问题的常用方法。它们降低了生产级软件中模型表现不佳的风险。虽然至关重要，但用于学习异构回归集成的方法尚未大规模提出，而在经典ML文献中，堆叠，级联和投票大多仅限于分类问题。回归带来了不同的学习挑战，可能导致性能不佳，即使使用良好的同质集成模式，如bagging或boosting。在本文中，我们介绍了MetaBags，这是一种新颖的，实用的回归堆栈框架。MetaBags是一种元学习算法，它学习一组Meta决策树，旨在为每个查询选择一个基本模型（即专家），并专注于归纳偏差减少。一组Meta决策树是使用不同类型的元特征来学习的，这些元特征是专门为此目的而创建的，然后在元级别被打包。这个过程的目的是学习一个模型与公平的偏差方差权衡，其改善基础模型的性能与不同的专家对特定的输入空间子区域的预测多样性。所提出的方法和元特征被设计成使得它们即使在可用的训练数据中没有充分表示的空间的子区域中也能够实现良好的预测性能。对该方法进行了详尽的实证测试，评估了合成，开放和真实世界的应用程序数据集上的泛化误差和方法的可扩展性。所获得的结果表明，我们的方法显着优于现有的最先进的方法。
英文摘要：
Ensembles are popular methods for solving practical supervised learning problems. They reduce the risk of having underperforming models in production-grade software. Although critical, methods for learning heterogeneous regression ensembles have not been proposed at large scale, whereas in classical ML literature, stacking, cascading and voting are mostly restricted to classification problems. Regression poses distinct learning challenges that may result in poor performance, even when using well established homogeneous ensemble schemas such as bagging or boosting. In this paper, we introduce MetaBags, a novel, practically useful stacking framework for regression. MetaBags is a meta-learning algorithm that learns a set of meta-decision trees designed to select one base model (i.e. expert) for each query, and focuses on inductive bias reduction. A set of meta-decision trees are learned using different types of meta-features, specially created for this purpose - to then be bagged at meta-level. This procedure is designed to learn a model with a fair bias-variance trade-off, and its improvement over base model performance is correlated with the prediction diversity of different experts on specific input space subregions. The proposed method and meta-features are designed in such a way that they enable good predictive performance even in subregions of space which are not adequately represented in the available training data. An exhaustive empirical testing of the method was performed, evaluating both generalization error and scalability of the approach on synthetic, open and real-world application datasets. The obtained results show that our method significantly outperforms existing state-of-the-art approaches.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
