arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1709.01177stat.MLcs.LG
收藏

记忆约束下随机树子空间的特征选择

Random Subspace with Trees for Feature Selection Under Memory Constraints

University of Liège(列日大学);Aix-Marseille University(艾克斯-马赛大学);New York University(纽约大学)

Antonio Sutera, Célia Châtel, Gilles Louppe, Louis Wehenkel, Pierre Geurts

摘要:

处理高维数据集是机器学习的一个主要挑战。在本文中,我们考虑的问题,在内存不够大,包含所有功能的应用程序中的功能选择。在这种情况下,我们提出了一种新的基于树的特征选择方法,建立一个序列的随机树上的小样本的变量混合两个变量已经确定为相关的以前的模型和变量随机选择的其他变量。作为我们的主要贡献,我们提供了一个深入的理论分析,这种方法在无限样本设置。特别是,我们研究其合理性方面的常见定义的功能相关性和收敛速度在各种变量依赖的情况下。我们还提供了一些初步的实证结果,突出了这种方法的潜力。

英文摘要:

Dealing with datasets of very high dimension is a major challenge in machine learning. In this paper, we consider the problem of feature selection in applications where the memory is not large enough to contain all features. In this setting, we propose a novel tree-based feature selection approach that builds a sequence of randomized trees on small subsamples of variables mixing both variables already identified as relevant by previous models and variables randomly selected among the other variables. As our main contribution, we provide an in-depth theoretical analysis of this method in infinite sample setting. In particular, we study its soundness with respect to common definitions of feature relevance and its convergence speed under various variable dependance scenarios. We also provide some preliminary empirical results highlighting the potential of the approach.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑