arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2410.02344cs.LG
收藏

EntryPrune:基于第一印象的神经网络特征选择

EntryPrune: Neural Network Feature Selection using First Impressions

ETH Zurich(苏黎世联邦理工学院)

Felix Zimmer, Patrik Okanovic, Torsten Hoefler

AI总结:

本文提出EntryPrune,一种基于动态稀疏输入层和条目剪枝的神经网络特征选择算法,在13个数据集上优于现有方法,尤其提升低维数据准确率并降低运行时间。

摘要:

目前,人们正在努力开发特征选择算法,以提高预测模型的可解释性、减少计算资源消耗并最小化过拟合。神经网络作为构建特征选择方法的架构脱颖而出,最近,神经元剪枝和再生从稀疏神经网络文献中涌现出来,成为有前景的新工具。我们提出了EntryPrune,一种新颖的有监督特征选择算法,使用具有动态稀疏输入层的密集神经网络。它采用基于条目(entry-based)的剪枝,这是一种新颖的方法,根据神经元进入网络时引起的相对变化来比较神经元。在13个不同数据集上的大量实验表明,我们的方法通常优于当前最先进的方法,特别是在低维数据集上提高了平均准确率。此外,我们展示了在EntryPrune框架内,EntryPruning超越了诸如幅度剪枝等传统技术,并且EntryPrune比竞争方法实现了更低的运行时间。我们的代码可在https://github.com/flxzimmer/entryprune获取。

英文摘要:

There is an ongoing effort to develop feature selection algorithms to improve interpretability, reduce computational resources, and minimize overfitting in predictive models. Neural networks stand out as architectures on which to build feature selection methods, and recently, neuron pruning and regrowth have emerged from the sparse neural network literature as promising new tools. We introduce EntryPrune, a novel supervised feature selection algorithm using a dense neural network with a dynamic sparse input layer. It employs entry-based pruning, a novel approach that compares neurons based on their relative change induced when they have entered the network. Extensive experiments on 13 different datasets show that our approach generally outperforms the current state-of-the-art methods, and in particular improves the average accuracy on low-dimensional datasets. Furthermore, we show that EntryPruning surpasses traditional techniques such as magnitude pruning within the EntryPrune framework and that EntryPrune achieves lower runtime than competing approaches. Our code is available at https://github.com/flxzimmer/entryprune.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑