Variable importance plays a pivotal role in interpretable machine learning as it helps measure the impact of factors on the output of the prediction model. Model agnostic methods based on the generation of "null" features via permutation (or related approaches) can be applied. Such analysis is often utilized in pharmaceutical applications due to its ability to interpret black-box models, including tree-based ensembles. A major challenge and significant confounder in variable importance estimation however is the presence of between-feature correlation. Recently, several adjustments to marginal permutation utilizing feature knockoffs were proposed to address this issue, such as the variable importance measure known as conditional predictive impact (CPI). Assessment and evaluation of such approaches is the focus of our work. We first present a comprehensive simulation study investigating the impact of feature correlation on the assessment of variable importance. We then theoretically prove the limitation that highly correlated features pose for the CPI through the knockoff construction. While we expect that there is always no correlation between knockoff variables and its corresponding predictor variables, we prove that the correlation increases linearly beyond a certain correlation threshold between the predictor variables. Our findings emphasize the absence of free lunch when dealing with high feature correlation, as well as the necessity of understanding the utility and limitations behind methods in variable importance estimation.
Statistical Significance of Feature Importance Rankings
Department of Statistics University of California, Berkeley（统计学系加州大学伯克利分校）；Department of Statistics and Data Science University of Pennsylvania（统计学与数据科学系宾夕法尼亚大学）
本文针对特征重要性排名因随机采样而不稳定的问题，提出基于假设检验的验证方法和两种高效采样算法，能以超过 1-α 的概率正确识别前 K 个重要特征及其顺序，并在 SHAP 和 LIME 上验证了有效性。
Variable Importance Clouds: A Way to Explore Variable Importance for the Set of Good Models
Department of Economics, Duke University（杜克大学经济学系）；Departments of Computer Science, Electrical and Computer Engineering, and Statistical Science, Duke University（杜克大学计算机科学系、计算机与电子工程系和统计科学系）；Duke University（杜克大学）
Comparing Explanation Methods for Traditional Machine Learning Models Part 2: Quantifying Model Explainability Faithfulness and Improvements with Dimensionality Reduction
Cooperative Institute for Severe and High-Impact Weather Research and Operations, University of Oklahoma（俄克拉荷马大学强天气与高影响天气合作研究所）；University of Oklahoma（俄克拉荷马大学）；NOAA/OAR/National Severe Storms Laboratory（美国国家海洋和大气管理局/海洋与大气研究办公室/国家强风暴实验室）；Verisk Analytics Incorporated（Verisk分析公司）；NSF AI Institute for Research on Trustworthy AI in Weather, Climate, and Coastal Oceanography（美国国家科学基金会天气、气候与沿海海洋学可信人工智能研究人工智能研究所）
FiBiNET: Combining Feature Importance and Bilinear feature Interaction for Click-Through Rate Prediction
Sina Weibo Inc.（新浪微博公司）
广告和提要排名对Facebook和新浪微博等许多互联网公司至关重要。在许多现实世界的广告和提要排名系统中，点击率（CTR）预测起着核心作用。在该领域中有许多提出的模型，例如逻辑回归、基于树的模型、基于因子分解机的模型和基于深度学习的CTR模型。然而，目前的许多工作都是以简单的方式计算特征的相互作用，如Hadamard积和内积，他们很少关心特征的重要性。提出了一种新的特征重要性和双线性特征交互网络（FeatureImportance and BilinearfeatureInteractionNETwork，简称FiBiNET）模型，用于动态学习特征重要性和细粒度特征交互。一方面，FiBiNET可以通过挤压激励网络（SENET）机制动态学习特征的重要性;另一方面，它能够通过双线性函数有效地学习特征的相互作用。我们在两个真实世界的数据集上进行了广泛的实验，并表明我们的浅层模型优于其他浅层模型，如因子分解机（FM）和场感知因子分解机（FFM）。为了进一步提高性能，我们将经典的深度神经网络（DNN）组件与浅层模型相结合，成为深度模型。深度FiBiNET始终优于其他最先进的深度模型，如DeepFM和极端深度因子分解机（XdeepFM）。
Reconstructing dynamical networks via feature ranking
Faculty of Information Studies in Novo Mesto（新梅斯托信息研究学院）；Universitat Pompeu Fabra（庞培法布拉大学）；Jožef Stefan Institute（约瑟夫·斯特凡研究所）；University of Ljubljana（卢布尔雅那大学）
