We investigate feature selection problem for generic machine learning models. We introduce a novel framework that selects features considering the outcomes of the model. Our framework introduces a novel feature masking approach to eliminate the features during the selection process, instead of completely removing them from the dataset. This allows us to use the same machine learning model during feature selection, unlike other feature selection methods where we need to train the machine learning model again as the dataset has different dimensions on each iteration. We obtain the mask operator using the predictions of the machine learning model, which offers a comprehensive view on the subsets of the features essential for the predictive performance of the model. A variety of approaches exist in the feature selection literature. However, to our knowledge, no study has introduced a training-free framework for a generic machine learning model to select features while considering the importance of the feature subsets as a whole, instead of focusing on the individual features. We demonstrate significant performance improvements on the real-life datasets under different settings using LightGBM and Multi-Layer Perceptron as our machine learning models. The high performance of our General Binary Mask Optimization algorithm stems from its feature masking approach to select features and its flexibility in the number of selected features. The algorithm selects features based on the validation performance of the machine learning model. Hence, the number of selected features is not predetermined and adjusts dynamically to the dataset. Additionally, we openly share the implementation or our code to encourage further research in this area.
Multi-label feature selection based on binary hashing learning and dynamic graph constraints
Key Laboratory of Intelligent Education Technology and Application of Zhejiang Province Zhejiang Normal University(浙江省智能教育技术与应用重点实验室,浙江师范大学);School of Computing, Mathematics and Engineering, Charles Sturt University(查尔斯·斯特劳特大学计算机、数学与工程学院)
Optimizing Feature Selection for Binary Classification with Noisy Labels: A Genetic Algorithm Approach
A. I. Virtanen Institute for Molecular Sciences, University of Eastern Finland(A.I. Virtanen分子科学研究所,东芬兰大学);Department of Computer Science, Aalto University(计算机科学系,艾尔托大学);Institute of Biomedicine, University of Eastern Finland(生物医学研究所,东芬兰大学)
A Fuzzy-Rough based Binary Shuffled Frog Leaping Algorithm for Feature Selection
Memorial University of Newfoundland(纽芬兰纪念大学);University of Windsor(温莎大学);Shahid Bahonar University of Kerman(克尔曼沙希德·巴霍纳尔大学);Gwangju Institute of Science and Technology(光州科学技术院)
特征选择和属性约简是机器学习、数据采掘和模式识别领域中的关键问题,并且被广泛使用的技术通过选择特征的子集或去除不相关的特征来克服众所周知的Curse of Precisionality现象。本文提出了一种新的特征选择方法,有效地进行属性约简,从而选择数据集的信息量最大的特征。它由两个部分组成:1)一个功能子集评估的措施,和2)一个搜索策略。对于评价指标,我们采用了模糊粗糙依赖度(FRFDD)在基于下近似的模糊粗糙特征选择(L-FRFS),由于其有效性在特征选择。在搜索策略方面,提出了一种新的二进制混合蛙跳算法(B-SFLA)。将B-SFLA与FRDD相结合,得到了一种新的特征选择方法。非参数统计测试进行比较所提出的方法与现有的几种方法超过22个数据集,包括9个高维和大型的,从UCI存储库。实验结果表明,B-SFLA方法显着优于其他元启发式方法的选择功能的数量和分类精度。
SparseSwaps: Tractable LLM Pruning Mask Refinement at Scale
Department for AI in Society, Science, and Technology, Zuse Institute Berlin, Germany(人工智能与社会、科学与技术部门,柏林Zuse研究所,德国);Institute of Mathematics, Technische Universität Berlin, Germany(数学研究所,柏林技术大学,德国)
Forget-free Continual Learning with Soft-Winning SubNetworks
KAIST(韩国科学技术院)
受正则化彩票假说(RLTH)的启发,该假说指出,竞争性光滑(非二进制)子网络存在于一个密集的网络中的持续学习任务,我们研究了两个建议的基于架构的持续学习方法,顺序学习和选择自适应二进制(WSN)和非二进制软子网(SoftNet)为每个任务。WSN和SoftNet共同学习与每个任务相关联的子网络的正则化模型权重和任务自适应非二进制掩码,同时尝试通过重用先前子网络的权重来选择要激活的一小组权重(中奖票)。我们提出的无线传感器网络和软网络是固有的免疫灾难性遗忘,因为每个选定的子网络模型不侵犯其他子网络的任务增量学习(TIL)。在TIL中,每个中奖彩票产生的二进制掩码被编码成一个N位二进制数字掩码,然后使用霍夫曼编码进行压缩,以使网络容量与任务数量呈次线性增长。令人惊讶的是,在推理步骤中,通过向所获取的WSN的背景注入小噪声(保持WSN的前景)而生成的软网为TIL中的未来任务提供了出色的前向传输能力。SoftNet在Few-Shot Class Incremental Learning(FSCIL)中的一些示例中显示了其在正则化参数以解决过拟合方面优于WSN的有效性。
Learning to Rearrange Voxels in Binary Segmentation Masks for Smooth Manifold Triangulation
Graz University of Technology(格拉茨技术大学);Computer Algorithms for Medicine Laboratory (Café-Lab)(医学计算机算法实验室(Café-Lab));Medical University Graz(格拉茨医科大学);University Hospital Essen(埃森大学医院);ZhejiangLab(之江实验室)