arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2009.10306physics.comp-phphysics.flu-dyn
收藏

用于仿真控制的分布式深度强化学习

Distributed deep reinforcement learning for simulation control

Suraj Pawar, Romit Maulik

摘要:

物理系统的科学模拟中的一些应用可以用公式表示为控制/优化问题。这种系统的计算模型通常包含超参数,控制解决方案的保真度和计算费用。这些参数的调整是不平凡的,一般的方法是手动“抽查”好的组合。这是因为当参数空间很大并且它们可能动态变化时,最优超参数配置搜索变得不切实际。为了解决这个问题,我们提出了一个基于深度强化学习(RL)的框架来训练深度神经网络代理,该代理通过动态改变参数来控制模型求解。首先,我们验证了我们的RL框架的问题,控制混沌系统中的混沌动态改变系统的参数。随后,我们说明了我们的框架的能力,通过自动调整离散化纳维-斯托克斯方程的松弛因子在运行时加速收敛的稳态CFD求解器。结果表明,松弛因子的运行时控制的学习政策导致收敛的迭代次数显着减少相比,随机选择的松弛因子。我们的研究结果指出了在不同的几何形状和边界条件下学习自适应超参数学习策略的潜在好处,并对减少计算活动费用产生了影响。\footnote{Data and codes available at \url{https://github.com/Romit-Maulik/PAR-RL}}

英文摘要:

Several applications in the scientific simulation of physical systems can be formulated as control/optimization problems. The computational models for such systems generally contain hyperparameters, which control solution fidelity and computational expense. The tuning of these parameters is non-trivial and the general approach is to manually `spot-check' for good combinations. This is because optimal hyperparameter configuration search becomes impractical when the parameter space is large and when they may vary dynamically. To address this issue, we present a framework based on deep reinforcement learning (RL) to train a deep neural network agent that controls a model solve by varying parameters dynamically. First, we validate our RL framework for the problem of controlling chaos in chaotic systems by dynamically changing the parameters of the system. Subsequently, we illustrate the capabilities of our framework for accelerating the convergence of a steady-state CFD solver by automatically adjusting the relaxation factors of discretized Navier-Stokes equations during run-time. The results indicate that the run-time control of the relaxation factors by the learned policy leads to a significant reduction in the number of iterations for convergence compared to the random selection of the relaxation factors. Our results point to potential benefits for learning adaptive hyperparameter learning strategies across different geometries and boundary conditions with implications for reduced computational campaign expenses. \footnote{Data and codes available at \url{https://github.com/Romit-Maulik/PAR-RL}}

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑