arXiv 2302.103442023-02-20math.OCcs.LG

基于模型的神经网络特征选择：一种混合整数规划方法

Model-based feature selection for neural networks: A mixed-integer programming approach

KTH Royal Institute of Technology（瑞典皇家理工学院）；Imperial College London（帝国理工学院）

Shudian Zhao, Calvin Tsay, Jan Kronqvist

备注：: 15 pages, 3 figures, 5 tables

摘要：

在这项工作中，我们为基于ReLU的深度神经网络（DNN）开发了一种新的输入特征选择框架，该框架建立在混合整数优化方法的基础上。虽然该方法通常适用于各种分类任务，但我们专注于寻找用于图像分类的输入特征，以便清晰地呈现。这个想法是使用一个经过训练的DNN或一组经过训练的DNN来识别显著的输入特征。将输入特征选择公式化为一系列混合整数线性规划（MILP）问题，该问题找到使每个类别的分类置信度最大化的稀疏输入集。这些“逆”问题通过为每个类别选择的输入数量和分布约束进行正则化。在著名的MNIST和FashionMNIST数据集上的数值结果表明，所提出的输入特征选择允许我们将输入的大小大幅减少到$\sim$ 15\%，同时保持良好的分类精度。这使我们能够设计具有更少连接的DNN，减少计算工作量，并产生对对抗性攻击更鲁棒的DNN。

英文摘要：

In this work, we develop a novel input feature selection framework for ReLU-based deep neural networks (DNNs), which builds upon a mixed-integer optimization approach. While the method is generally applicable to various classification tasks, we focus on finding input features for image classification for clarity of presentation. The idea is to use a trained DNN, or an ensemble of trained DNNs, to identify the salient input features. The input feature selection is formulated as a sequence of mixed-integer linear programming (MILP) problems that find sets of sparse inputs that maximize the classification confidence of each category. These ''inverse'' problems are regularized by the number of inputs selected for each category and by distribution constraints. Numerical results on the well-known MNIST and FashionMNIST datasets show that the proposed input feature selection allows us to drastically reduce the size of the input to $\sim$15\% while maintaining a good classification accuracy. This allows us to design DNNs with significantly fewer connections, reducing computational effort and producing DNNs that are more robust towards adversarial attacks.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

摘要：

英文摘要：

相关论文