表格生物医学数据中低性能机器学习特征重要性的有效性
Validity of Feature Importance in Low-Performing Machine Learning for Tabular Biomedical Data
AI总结：
本研究挑战低性能模型不能用于特征重要性分析的传统观点，通过合成和真实生物医学数据实验，证明在数据量充足时，特征切割下低性能模型仍能保持特征排名的有效性。
摘要：
在表格生物医学数据分析中，将模型调优至高准确率被认为是讨论特征重要性的前提，因为医学从业者期望特征重要性的有效性与性能相关。在本工作中，我们挑战了这一普遍信念，表明低性能模型也可用于特征重要性分析。我们提出了实验来观察随着性能逐步下降，特征排名的变化。使用三个合成数据集和六个真实生物医学数据集，我们将来自完整数据集的特征排名与样本量减少（数据切割）或特征减少（特征切割）后的数据集进行比较。在合成数据集中，特征切割不改变特征排名，而数据切割在较低性能下显示出更高的差异。在真实数据集中，特征切割显示的变化与数据切割相似或更小，尽管某些数据集表现出相反的情况。当通过去除相关性来控制特征交互时，特征切割始终显示出更好的稳定性。通过分析特征重要性值的分布并从理论上检验模型无法区分特征之间特征重要性的概率，我们揭示模型在特征切割下即使性能下降仍能区分特征重要性，但在数据切割下则不能。我们得出结论，如果数据量充足，即使在低性能水平下，特征重要性的有效性也可以保持，这是导致表格医学数据分析中性能欠佳的一个重要因素。本文展示了即使在分类器性能不令人满意时，利用特征重要性分析与统计分析相结合来相对比较特征的潜力。
英文摘要：
In tabular biomedical data analysis, tuning models to high accuracy is considered a prerequisite for discussing feature importance, as medical practitioners expect the validity of feature importance to correlate with performance. In this work, we challenge the prevailing belief, showing that low-performing models may also be used for feature importance. We propose experiments to observe changes in feature rank as performance degrades sequentially. Using three synthetic datasets and six real biomedical datasets, we compare the rank of features from full datasets to those with reduced sample sizes (data cutting) or fewer features (feature cutting). In synthetic datasets, feature cutting does not change feature rank, while data cutting shows higher discrepancies with lower performance. In real datasets, feature cutting shows similar or smaller changes than data cutting, though some datasets exhibit the opposite. When feature interactions are controlled by removing correlations, feature cutting consistently shows better stability. By analyzing the distribution of feature importance values and theoretically examining the probability that the model cannot distinguish feature importance between features, we reveal that models can still distinguish feature importance despite performance degradation through feature cutting, but not through data cutting. We conclude that the validity of feature importance can be maintained even at low performance levels if the data size is adequate, which is a significant factor contributing to suboptimal performance in tabular medical data analysis. This paper demonstrates the potential for utilizing feature importance analysis alongside statistical analysis to compare features relatively, even when classifier performance is not satisfactory.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
