arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1902.11074cs.LGcs.AIstat.ML
收藏

AFS:一种基于注意力的监督特征选择机制

AFS: An Attention-based mechanism for Supervised Feature Selection

Central South University(中南大学);Zhejiang Sci-Tech University(浙江理工大学)

Ning Gui, Danni Ge, Ziyin Hu

摘要:

特征选择作为一个有效的数据预处理步骤,已经显示出它的有效性,准备高维数据的许多机器学习任务。然而,高维海量数据的激增给现有的特征选择技术带来了巨大的挑战,如计算复杂度和噪声数据下的稳定性。本文介绍了一种新的基于神经网络的特征选择结构,称为基于注意力的特征选择(AFS)。AFS由两个可分离的模块组成:用于特征权重生成的注意模块和用于问题建模的学习模块。注意力模块将特征与监控目标之间的关联问题转化为一个二元分类问题,每个特征由一个浅层注意力网络支持。在训练过程中,基于通过反向传播调整的各个特征选择模式的分布来生成特征权重。可拆卸的结构允许直接重用现有的现成模型,这允许更少的训练时间、对训练数据的需求和对专业知识的要求。为了提高特征选择的准确性,本文还提出了一种混合初始化方法,以解决样本量不足的情况下特征选择的问题。实验结果表明,AFS算法在MNIST、噪声MNIST和小样本数据集上均获得了最佳的准确性和稳定性,优于现有的几种特征选择算法.

英文摘要:

As an effective data preprocessing step, feature selection has shown its effectiveness to prepare high-dimensional data for many machine learning tasks. The proliferation of high di-mension and huge volume big data, however, has brought major challenges, e.g. computation complexity and stability on noisy data, upon existing feature-selection techniques. This paper introduces a novel neural network-based feature selection architecture, dubbed Attention-based Feature Selec-tion (AFS). AFS consists of two detachable modules: an at-tention module for feature weight generation and a learning module for the problem modeling. The attention module for-mulates correlation problem among features and supervision target into a binary classification problem, supported by a shallow attention net for each feature. Feature weights are generated based on the distribution of respective feature se-lection patterns adjusted by backpropagation during the train-ing process. The detachable structure allows existing off-the-shelf models to be directly reused, which allows for much less training time, demands for the training data and requirements for expertise. A hybrid initialization method is also intro-duced to boost the selection accuracy for datasets without enough samples for feature weight generation. Experimental results show that AFS achieves the best accuracy and stability in comparison to several state-of-art feature selection algo-rithms upon both MNIST, noisy MNIST and several datasets with small samples.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑