间接参数化的Concrete Autoencoders
Indirectly Parameterized Concrete Autoencoders
AI总结：
针对Concrete自编码器特征选择时因重复特征选择导致联合优化不稳定、训练慢且泛化差的问题，提出间接参数化的IP-CAE，通过学习嵌入到Gumbel-Softmax参数的映射，显著提升泛化能力与训练效率，还可推广至相关分布场景。
摘要：
在数据高维或获取完整特征集成本高昂的场景中，特征选择是一项关键任务。基于神经网络的嵌入式特征选择领域的最新进展在各类应用中展现出可观成效。被视为嵌入式特征选择当前最优方法的Concrete Autoencoders（CAE，Concrete自编码器）可能难以实现稳定的联合优化，进而影响其训练时长与泛化能力。在本研究中，我们发现这种不稳定性与CAE学习到重复选择的特征相关。为解决这一问题，我们提出一种简单有效的改进方案：Indirectly Parameterized CAEs（IP-CAE，间接参数化Concrete自编码器）。IP-CAE学习一个嵌入向量，以及从该嵌入向量到Gumbel-Softmax分布参数的映射。尽管实现简单，在多个数据集的重构与分类任务中，IP-CAE在泛化能力和训练时长上均较CAE展现出显著且一致的提升。与CAE不同，IP-CAE能有效利用非线性关系，且无需重新训练联合优化的解码器。此外，我们的方法原则上可推广至特征选择之外的Gumbel-Softmax分布相关场景。
英文摘要：
Feature selection is a crucial task in settings where data is high-dimensional or acquiring the full set of features is costly. Recent developments in neural network-based embedded feature selection show promising results across a wide range of applications. Concrete Autoencoders (CAEs), considered state-of-the-art in embedded feature selection, may struggle to achieve stable joint optimization, hurting their training time and generalization. In this work, we identify that this instability is correlated with the CAE learning duplicate selections. To remedy this, we propose a simple and effective improvement: Indirectly Parameterized CAEs (IP-CAEs). IP-CAEs learn an embedding and a mapping from it to the Gumbel-Softmax distributions' parameters. Despite being simple to implement, IP-CAE exhibits significant and consistent improvements over CAE in both generalization and training time across several datasets for reconstruction and classification. Unlike CAE, IP-CAE effectively leverages non-linear relationships and does not require retraining the jointly optimized decoder. Furthermore, our approach is, in principle, generalizable to Gumbel-Softmax distributions beyond feature selection.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
