arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2403.02730cs.LGcs.CEmath.OC
收藏

一种用于用神经网络建模约束系统的两阶段训练方法

A Two-Stage Training Method for Modeling Constrained Systems With Neural Networks

University of Minho(米尼奥大学);University of Porto(波尔图大学)

C. Coelho, M. Fernanda P. Costa, L. L. Ferrás

AI总结:

针对现有带约束神经网络训练需手动调超参、约束融入效果存疑的问题,提出一种适用于任意NN架构的Neural ODE两阶段训练法,分阶段找可行参数与最优解,实验证明其能满足约束、提升预测性能与收敛性、增强可解释性并降低数据需求。

摘要:

现实世界中的系统通常被建模为约束优化问题。目前已有将约束融入神经网络(NN)的技术得到应用,例如神经常微分方程(Neural ODEs)。但这类方法会引入需要通过反复试错手动调优的超参数,使得人们对生成模型是否成功融入约束产生质疑。本文详细介绍了针对Neural ODEs的两阶段训练方法,这是一种简单有效、无需惩罚参数的约束系统建模范式。该方法将约束优化问题重构为两个无约束子问题,分两个阶段求解:第一阶段通过最小化约束违反程度的度量,寻找可行的神经网络参数;第二阶段在保持参数处于可行域内的前提下,最小化损失函数以找到最优神经网络参数。我们通过实验证明,该方法生成的模型既能满足约束,还能提升预测性能,既确保了关键系统属性的合规性,也有助于降低数据量需求。此外,研究表明该方法能提升向最优解收敛的效果,还能增强Neural ODE模型的可解释性。本文提出的两阶段训练方法可适配任意神经网络架构。

英文摘要:

Real-world systems are often formulated as constrained optimization problems. Techniques to incorporate constraints into Neural Networks (NN), such as Neural Ordinary Differential Equations (Neural ODEs), have been used. However, these introduce hyperparameters that require manual tuning through trial and error, raising doubts about the successful incorporation of constraints into the generated model. This paper describes in detail the two-stage training method for Neural ODEs, a simple, effective, and penalty parameter-free approach to model constrained systems. In this approach the constrained optimization problem is rewritten as two unconstrained sub-problems that are solved in two stages. The first stage aims at finding feasible NN parameters by minimizing a measure of constraints violation. The second stage aims to find the optimal NN parameters by minimizing the loss function while keeping inside the feasible region. We experimentally demonstrate that our method produces models that satisfy the constraints and also improves their predictive performance. Thus, ensuring compliance with critical system properties and also contributing to reducing data quantity requirements. Furthermore, we show that the proposed method improves the convergence to an optimal solution and improves the explainability of Neural ODE models. Our proposed two-stage training method can be used with any NN architectures.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑