新兴变量重要性估计方法的计算探索
A Computational Exploration of Emerging Methods of Variable Importance Estimation
摘要：
估计变量的重要性是现代机器学习中的一项基本任务。这有助于评估给定模型中特征的优良性。在过去十年中，已经开发了几种用于估计变量重要性的技术。在本文中，我们提出了一个计算和理论探索的新兴方法的变量的重要性估计，即：最小绝对收缩和选择算子（LASSO），支持向量机（SVM），预测误差函数（PERF），随机森林（RF）和极端梯度提升（XGBOOST）进行了测试，对不同种类的现实生活和模拟数据。所有这些方法都可以无缝地处理回归和分类任务，但在处理包含缺失值的数据时都失败了。实现表明，PERF具有最好的性能，在高度相关的数据的情况下，紧随其后的RF。PERF和XGBOOST是“数据饥渴”的方法，它们在小数据量上的性能最差，但在执行时间上它们是最快的。当数据集中存在大量冗余特征时，SVM是最合适的。PERF的盈余是其在零处的自然截止值，这有助于将正面和负面分数分开，所有正面分数表示基本和重要的特征，而负面分数表示无用的特征。RF和LASSO非常通用，几乎可以在所有情况下使用，尽管它们没有给出最佳结果。
英文摘要：
Estimating the importance of variables is an essential task in modern machine learning. This help to evaluate the goodness of a feature in a given model. Several techniques for estimating the importance of variables have been developed during the last decade. In this paper, we proposed a computational and theoretical exploration of the emerging methods of variable importance estimation, namely: Least Absolute Shrinkage and Selection Operator (LASSO), Support Vector Machine (SVM), the Predictive Error Function (PERF), Random Forest (RF), and Extreme Gradient Boosting (XGBOOST) that were tested on different kinds of real-life and simulated data. All these methods can handle both regression and classification tasks seamlessly but all fail when it comes to dealing with data containing missing values. The implementation has shown that PERF has the best performance in the case of highly correlated data closely followed by RF. PERF and XGBOOST are "data-hungry" methods, they had the worst performance on small data sizes but they are the fastest when it comes to the execution time. SVM is the most appropriate when many redundant features are in the dataset. A surplus with the PERF is its natural cut-off at zero helping to separate positive and negative scores with all positive scores indicating essential and significant features while the negatives score indicates useless features. RF and LASSO are very versatile in a way that they can be used in almost all situations despite they are not giving the best results.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
