arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2401.12644cs.LG
收藏

面向特征选择的二元特征掩码优化

Binary Feature Mask Optimization for Feature Selection

Bilkent University(毕尔肯大学)

Mehmet E. Lorasdagi, Mehmet Y. Turali, Suleyman S. Kozat

AI总结:

针对通用机器学习模型的特征选择问题,提出免训练的通用二元掩码优化框架,通过特征掩码动态选择特征子集,在真实数据集上实现显著性能提升并开源代码。

摘要:

我们针对通用机器学习模型研究特征选择问题,提出了一种结合模型输出结果进行特征选择的新型框架。该框架引入了全新的特征掩码方法,在选择过程中通过掩码屏蔽特征,而非将其从数据集中完全移除,这使得我们在特征选择阶段可使用同一个机器学习模型;不同于其他特征选择方法因每次迭代数据集维度不同,需要重新训练模型。我们利用机器学习模型的预测结果得到掩码算子,可全面掌握对模型预测性能至关重要的特征子集的整体情况。\n现有特征选择领域已有多种研究方法,但据我们所知,尚无研究针对通用机器学习模型提出免训练的特征选择框架,且该框架关注特征子集的整体重要性,而非单个特征的重要性。我们以LightGBM和多层感知机作为机器学习模型,在不同设置下的真实数据集上验证了该方法可带来显著的性能提升。我们提出的General Binary Mask Optimization(通用二元掩码优化)算法的高性能源于其采用的特征掩码选择方式,以及在所选特征数量上的灵活性:该算法基于机器学习模型的验证性能选择特征,因此所选特征的数量并非预先设定,而是根据数据集动态调整。此外,我们公开共享了代码实现,以推动该领域的进一步研究。

英文摘要:

We investigate feature selection problem for generic machine learning models. We introduce a novel framework that selects features considering the outcomes of the model. Our framework introduces a novel feature masking approach to eliminate the features during the selection process, instead of completely removing them from the dataset. This allows us to use the same machine learning model during feature selection, unlike other feature selection methods where we need to train the machine learning model again as the dataset has different dimensions on each iteration. We obtain the mask operator using the predictions of the machine learning model, which offers a comprehensive view on the subsets of the features essential for the predictive performance of the model. A variety of approaches exist in the feature selection literature. However, to our knowledge, no study has introduced a training-free framework for a generic machine learning model to select features while considering the importance of the feature subsets as a whole, instead of focusing on the individual features. We demonstrate significant performance improvements on the real-life datasets under different settings using LightGBM and Multi-Layer Perceptron as our machine learning models. The high performance of our General Binary Mask Optimization algorithm stems from its feature masking approach to select features and its flexibility in the number of selected features. The algorithm selects features based on the validation performance of the machine learning model. Hence, the number of selected features is not predetermined and adjusts dynamically to the dataset. Additionally, we openly share the implementation or our code to encourage further research in this area.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑