Feature selection in machine learning is subject to the intrinsic randomness of the feature selection algorithms (for example, random permutations during MDA). Stability of selected features with respect to such randomness is essential to the human interpretability of a machine learning algorithm. We proposes a rank based stability metric called instability index to compare the stabilities of three feature selection algorithms MDA, LIME, and SHAP as applied to random forests. Typically, features are selected by averaging many random iterations of a selection algorithm. Though we find that the variability of the selected features does decrease as the number of iterations increases, it does not go to zero, and the features selected by the three algorithms do not necessarily converge to the same set. We find LIME and SHAP to be more stable than MDA, and LIME is at least as stable as SHAP for the top ranked features. Hence overall LIME is best suited for human interpretability. However, the selected set of features from all three algorithms significantly improves various predictive metrics out of sample, and their predictive performances do not differ significantly. Experiments were conducted on synthetic datasets, two public benchmark datasets, and on proprietary data from an active investment strategy.
Dual-Directed Algorithm Design for Efficient Pure Exploration
Stanford Graduate School of Business, Stanford University（斯坦福商学院、斯坦福大学）；Department of Industrial Engineering and Decision Analytics, The Hong Kong University of Science and Technology（工业工程与决策分析系、香港科学与技术大学）
Institute for Data, Systems, and Society（数据、系统与社会研究所）；School of Data Science（数据科学学院）；Department of Decision Analytics and Operations（决策分析与运营部门）；Department of Civil and Environmental Engineering（土木与环境工程部门）；Operations Research Center（运筹学研究中心）
Selection of the Best Policy under Fairness Constraints for Subpopulations
Department of Industrial Engineering and Operations Research（工业工程与运营管理系）；Berkeley Artificial Intelligence Research Lab（伯克利人工智能研究实验室）；University of California Berkeley（加州大学伯克利分校）
A Splicing Approach to Best Subset of Groups Selection
Renmin University of China（中国人民大学）；Sun Yat-Sen University（中山大学）；National University of Singapore（新加坡国立大学）；University of Science and Technology of China（中国科学技术大学）
abess: A Fast Best Subset Selection Library in Python and R
Sun Yat-Sen University（中山大学）；University of Science and Technology of China（中国科学技术大学）；Renmin University of China（中国人民大学）；Peking University（北京大学）；National University of Singapore（新加坡国立大学）
Best-Subset Selection in Generalized Linear Models: A Fast and Consistent Algorithm via Splicing Technique
Saw Swee Hock School of Public Health, National University of Singapore（新加坡国立大学杨潞龄公共卫生学院）；Southern China Center for Statistical Science, Department of Statistical Science, School of Mathematics, Sun Yat-Sen University（中山大学数学学院统计科学系华南统计科学研究中心）；Department of Statistics and Finance/International Institute of Finance, School of Management, University of Science and Technology of China（中国科学技术大学管理学院统计与金融系/国际金融研究院）；School of Statistics and Information, Shanghai University of International Business and Economics（上海对外经贸大学统计与信息学院）
Nested cross-validation when selecting classifiers is overzealous for most practical applications
University of Campinas（坎皮纳斯大学）；University of East Anglia（东英吉利大学）
当选择要应用于特定问题的分类算法时，必须同时选择该数据集的最佳算法\emph{and}和所选模型的最佳超参数集。通常的方法是应用嵌套交叉验证过程;在内部交叉验证中执行超参数选择，而外部交叉验证计算算法\emph{with cross-validation based hyperparameter tuning}的预期准确度的无偏估计。另一种方法，我们称之为“平面交叉验证”，使用一个单一的交叉验证步骤来选择最佳超参数值，并提供算法的预期准确度的估计，尽管有偏差，但仍然可以用于选择最佳学习算法。我们使用12种不同的算法对115个真实生活中的二进制数据集进行了测试，并得出结论，使用计算成本较低的平面交叉验证程序通常会导致选择一种算法，该算法对于所有实际目的而言，与通过嵌套交叉验证选择的算法具有相似的质量，前提是学习算法具有相对较少的超参数进行优化。
BoSS: A Best-of-Strategies Selector as an Oracle for Deep Active Learning
Intelligent Embedded Systems University of Kassel（智能嵌入式系统卡塞尔大学）；Machine Learning Research Unit TU Wien（机器学习研究单位维也纳技术大学）；University of Kassel（卡塞尔大学）；TU Wien（维也纳技术大学）
Automatic selection of the best neural architecture for time series forecasting
Division of Applied Mathematics, Brown University（布朗大学应用数学系）；School of Engineering, Brown University（布朗大学工程学院）；Department of Mechanical Engineering, Massachusetts Institute of Technology（麻省理工学院机械工程系）
When the Strongest Teacher Is Not the Best Teacher: Student-Centric Answer Selection
University of Washington（华盛顿大学）；University of Texas at Austin（德克萨斯大学奥斯汀分校）；University of Southern California（南加州大学）；Independent Researcher（独立研究者）；National University of Singapore（新加坡国立大学）；Microsoft（微软）；Google（谷歌）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）；Northwestern University（西北大学）；Allen Institute for AI (AI2)（人工智能研究院（AI2））
