arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2202.11170cs.LGphysics.flu-dyn
收藏

用于形状优化的多保真度强化学习框架

Multi-fidelity reinforcement learning framework for shape optimization

University of Michigan(密歇根大学);Oklahoma State University(俄克拉荷马州立大学);Argonne National Laboratory(阿贡国家实验室)

Sahil Bhola, Suraj Pawar, Prasanna Balaprakash, Romit Maulik

AI总结:

本文提出一种利用多保真度仿真的可控迁移学习强化学习框架,解决了传统DRL对回合数需求高的问题,并在高雷诺数翼型形状优化中降低超30%计算成本,同时提升策略泛化能力。

摘要:

深度强化学习(DRL)是一种极具前景的外环智能范式,能够为复杂任务部署问题求解策略。因此,DRL已被应用于多个科学领域,特别是在经典优化或控制方法受限的情况下。传统DRL方法的一个关键局限在于其对回合数的极度渴求,这对于涉及数值正向模型昂贵评估的任务构成了瓶颈。本文通过引入一种利用多保真度仿真设置的可控迁移学习框架,解决了DRL的这一局限。我们的策略被应用于高雷诺数下的翼型形状优化问题,其中我们的框架能够通过从多保真度环境中收集知识,学习生成高效翼型形状的最优策略,并将计算成本降低超过30%。此外,我们的公式促进了策略探索以及对新环境的泛化能力,从而防止仅对单一保真度的数据产生过拟合。我们的结果证明了该框架对其他科学DRL场景的适用性,在这些场景中,多保真度环境可用于策略学习。

英文摘要:

Deep reinforcement learning (DRL) is a promising outer-loop intelligence paradigm which can deploy problem solving strategies for complex tasks. Consequently, DRL has been utilized for several scientific applications, specifically in cases where classical optimization or control methods are limited. One key limitation of conventional DRL methods is their episode-hungry nature which proves to be a bottleneck for tasks which involve costly evaluations of a numerical forward model. In this article, we address this limitation of DRL by introducing a controlled transfer learning framework that leverages a multi-fidelity simulation setting. Our strategy is deployed for an airfoil shape optimization problem at high Reynolds numbers, where our framework can learn an optimal policy for generating efficient airfoil shapes by gathering knowledge from multi-fidelity environments and reduces computational costs by over 30\%. Furthermore, our formulation promotes policy exploration and generalization to new environments, thereby preventing over-fitting to data from solely one fidelity. Our results demonstrate this framework's applicability to other scientific DRL scenarios where multi-fidelity environments can be used for policy learning.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑