arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2103.00329eess.SYcs.LGcs.SYphysics.flu-dyn
收藏

基于强化学习的湍流时变流中点到点导航的最优控制

Optimal control of point-to-point navigation in turbulent time-dependent flows using Reinforcement Learning

INFN, University of Rome Tor Vergata(罗马第二大学国家核物理研究所);Johns Hopkins University(约翰斯·霍普金斯大学);University of Gothenburg(哥德堡大学)

Michele Buzzicotti, Luca Biferale, Fabio Bonaccorso, Patricio Clark di Leoni, Kristian Gustavsson

摘要:

我们目前的理论和数值计算结果有关的问题,找到的路径,最大限度地减少在现实的导航约束下的复杂流体中的两个给定点之间的导航时间。我们对比确定性最优导航(ON)控制与强化学习(RL)算法获得的随机策略。我们表明,演员的评论RL算法能够找到准最优解的存在下,无论是时间无关的或混沌不断变化的流配置。对于我们的应用,ON解决方案在导航过程的典型持续时间内产生不稳定的行为,因此在实践中没有用。我们首先探讨使用恒定推进速度的湍流导航。基于离散相空间,调整推进方向,以最小化到达目标所花费的时间。此外,我们探讨的情况下,通过允许发动机断电获得额外的控制。利用底层流动的平流,允许以更少的能量消耗达到目标。在这种情况下,我们优化了总导航时间和发动机关闭总时间之间的线性组合。我们的方法可以推广到其他设置,例如,导航下不完善的环境预测或不同的模型的移动船舶。

英文摘要:

We present theoretical and numerical results concerning the problem to find the path that minimizes the time to navigate between two given points in a complex fluid under realistic navigation constraints. We contrast deterministic Optimal Navigation (ON) control with stochastic policies obtained by Reinforcement Learning (RL) algorithms. We show that Actor-Critic RL algorithms are able to find quasi-optimal solutions in the presence of either time-independent or chaotically evolving flow configurations. For our application, ON solutions develop unstable behavior within the typical duration of the navigation process, and are therefore not useful in practice. We first explore navigation of turbulent flow using a constant propulsion speed. Based on a discretized phase-space, the propulsion direction is adjusted with the aim to minimize the time spent to reach the target. Further, we explore a case where additional control is obtained by allowing the engine to power off. Exploiting advection of the underlying flow, allows the target to be reached with less energy consumption. In this case, we optimize a linear combination between the total navigation time and the total time the engine is switched off. Our approach can be generalized to other setups, for example, navigation under imperfect environmental forecast or with different models for the moving vessel.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑