arXivDaily arXiv每日学术速递 周一至周五更新
北美退伍军人节(Veterans Day),12 日无更新
arXiv 2101.09577cs.LGstat.ML
收藏

ReliefE:基于流形嵌入的高维空间特征排序

ReliefE: Feature Ranking in High-dimensional Spaces via Manifold Embeddings

Jožef Stefan Institute(约泽夫·斯蒂芬研究所);Jožef Stefan International Postgraduate School(约泽夫·斯蒂芬国际研究生院);University of Nova Gorica(新戈里察大学)

Blaž Škrlj, Sašo Džeroski, Nada Lavrač, Matej Petković

摘要:

特征排序已被广泛应用于高通量生物学和社会科学等机器学习应用中。流行的Relief算法家族的方法通过迭代地考虑最近的相关和不相关实例来分配重要性。尽管这些算法具有很高的实用性,但它们在计算上是昂贵的,并且不太适合高维稀疏输入空间。相比之下,最近基于嵌入的方法学习紧凑,低维的表示,潜在地促进传统学习器的下游学习能力。本文探讨了如何救济分支的算法可以适应受益于(黎曼)流形为基础的嵌入的实例和目标空间,其中一个给定的嵌入的维数是固有的所考虑的数据集的维数。开发的ReliefE算法速度更快,可以产生更好的特征排名,正如我们对20个真实数据集的多类和多标签分类任务的评估所示。ReliefE用于高维数据集的实用性是通过其利用稀疏矩阵代数运算的实现来确保的。最后,ReliefE与其他排序算法的关系进行了研究,通过模糊Jaccard指数。

英文摘要:

Feature ranking has been widely adopted in machine learning applications such as high-throughput biology and social sciences. The approaches of the popular Relief family of algorithms assign importances to features by iteratively accounting for nearest relevant and irrelevant instances. Despite their high utility, these algorithms can be computationally expensive and not-well suited for high-dimensional sparse input spaces. In contrast, recent embedding-based methods learn compact, low-dimensional representations, potentially facilitating down-stream learning capabilities of conventional learners. This paper explores how the Relief branch of algorithms can be adapted to benefit from (Riemannian) manifold-based embeddings of instance and target spaces, where a given embedding's dimensionality is intrinsic to the dimensionality of the considered data set. The developed ReliefE algorithm is faster and can result in better feature rankings, as shown by our evaluation on 20 real-life data sets for multi-class and multi-label classification tasks. The utility of ReliefE for high-dimensional data sets is ensured by its implementation that utilizes sparse matrix algebraic operations. Finally, the relation of ReliefE to other ranking algorithms is studied via the Fuzzy Jaccard Index.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑