arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2112.07007cs.LGcs.DSmath.OC
收藏

神经网络集成优化

Optimizing over an ensemble of neural networks

University of Connecticut(康涅狄格大学);University of Cincinnati(辛辛那提大学)

Keliang Wang, Leonardo Lozano, Carlos Cardonha, David Bergman

摘要:

我们研究优化问题,其中目标函数通过具有整流线性单元(ReLU)激活的前馈神经网络建模。最近的文献已经探索了使用单个神经网络来对目标函数内的不确定或复杂元素进行建模。然而,众所周知的是,集成的神经网络产生更稳定的预测,并具有更好的泛化能力比模型与单个神经网络,这促使研究集成的神经网络,而不是单个神经网络在决策管道。我们研究如何将一个神经网络集成作为优化模型的目标函数,并探索计算方法,随后的问题。我们提出了一个混合整数线性规划的基础上,现有的流行的大M配方优化一个单一的神经网络。我们为我们的模型开发了一个两阶段的方法,该方法结合了预处理程序,以收紧神经网络中关键神经元的边界,并采用基于拉格朗日松弛的分支定界方法。对我们的解决方案方法的实验评估表明,与单个神经网络相比,使用神经网络集合可以产生更稳定、更高质量的解决方案,并且我们的优化算法在计算时间和最优性差距方面优于最先进的方法。

英文摘要:

We study optimization problems where the objective function is modeled through feedforward neural networks with rectified linear unit (ReLU) activation. Recent literature has explored the use of a single neural network to model either uncertain or complex elements within an objective function. However, it is well known that ensembles of neural networks produce more stable predictions and have better generalizability than models with single neural networks, which motivates the investigation of ensembles of neural networks rather than single neural networks in decision-making pipelines. We study how to incorporate a neural network ensemble as the objective function of an optimization model and explore computational approaches for the ensuing problem. We present a mixed-integer linear program based on existing popular big-M formulations for optimizing over a single neural network. We develop a two-phase approach for our model that combines preprocessing procedures to tighten bounds for critical neurons in the neural networks with a Lagrangian relaxation-based branch-and-bound approach. Experimental evaluations of our solution methods suggest that using ensembles of neural networks yields more stable and higher quality solutions, compared to single neural networks, and that our optimization algorithm outperforms (the adaption of) a state-of-the-art approach in terms of computational time and optimality gaps.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑