arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2302.13221cs.LGcs.AI
收藏

超越离散选择:用于生成式特征选择的连续嵌入空间优化

Beyond Discrete Selection: Continuous Embedding Space Optimization for Generative Feature Selection

Computer Network Information Center, Chinese Academy of Sciences(中国科学院计算机网络信息中心);University of Chinese Academy of Sciences(中国科学院大学);University of Central Florida(中佛罗里达大学);Arizona State University(亚利桑那州立大学)

Meng Xiao, Dongjie Wang, Min Wu, Pengfei Wang, Yuanchun Zhou, Yanjie Fu

AI总结:

本文将特征选择重新表述为深度可微优化任务,提出连续嵌入空间优化框架,通过编码器-评估器-解码器结构和梯度上升搜索,解决高维小样本下的泛化与性能问题。

摘要:

特征选择——包括过滤式、包裹式和嵌入式方法——的目标是为指定的下游任务找到最优特征子集。然而,当前的特征选择方法受到以下限制:1)这些方法的选择标准因领域而异,使其难以泛化;2)当处理高维特征空间且样本量较小时,这些方法的选择性能显著下降。鉴于这些挑战,我们提出一个问题:所选的特征子集能否更鲁棒、更准确且对输入维度不敏感?在本文中,我们将特征选择问题重新表述为深度可微优化任务,并提出一种新的研究视角:将离散特征子集选择概念化为连续嵌入空间优化。我们引入了一个新颖且有原则的框架,包含序列编码器、准确率评估器、序列解码器和梯度上升优化器。这一综合框架包括四个重要步骤:特征-准确率训练数据的准备、深度特征子集嵌入、梯度优化搜索和特征子集重构。具体而言,我们利用强化特征选择学习来生成多样且高质量的训练数据,并增强泛化能力。通过优化重构和准确率损失,我们使用编码器-评估器-解码器模型结构将特征选择知识嵌入到连续空间中。我们采用梯度上升搜索算法在学习到的嵌入空间中寻找更好的嵌入。此外,我们利用这些嵌入重构特征选择解决方案,并为下游任务选择性能最高的特征子集作为最优子集。

英文摘要:

The goal of Feature Selection - comprising filter, wrapper, and embedded approaches - is to find the optimal feature subset for designated downstream tasks. Nevertheless, current feature selection methods are limited by: 1) the selection criteria of these methods are varied for different domains, making them hard to generalize; 2) the selection performance of these approaches drops significantly when processing high-dimensional feature space coupled with small sample size. In light of these challenges, we pose the question: can selected feature subsets be more robust, accurate, and input dimensionality agnostic? In this paper, we reformulate the feature selection problem as a deep differentiable optimization task and propose a new research perspective: conceptualizing discrete feature subsetting as continuous embedding space optimization. We introduce a novel and principled framework that encompasses a sequential encoder, an accuracy evaluator, a sequential decoder, and a gradient ascent optimizer. This comprehensive framework includes four important steps: preparation of features-accuracy training data, deep feature subset embedding, gradient-optimized search, and feature subset reconstruction. Specifically, we utilize reinforcement feature selection learning to generate diverse and high-quality training data and enhance generalization. By optimizing reconstruction and accuracy losses, we embed feature selection knowledge into a continuous space using an encoder-evaluator-decoder model structure. We employ a gradient ascent search algorithm to find better embeddings in the learned embedding space. Furthermore, we reconstruct feature selection solutions using these embeddings and select the feature subset with the highest performance for downstream tasks as the optimal subset.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑