arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2104.05437cs.LGnlin.CD
收藏

混沌时空动力学深度强化学习主动控制的对称性约简

Symmetry reduction for deep reinforcement learning active control of chaotic spatiotemporal dynamics

University of Wisconsin-Madison(威斯康星大学麦迪逊分校)

Kevin Zeng, Michael D. Graham

摘要:

深度强化学习(RL)是一种数据驱动、无模型的方法,能够发现高维系统中宏观目标的复杂控制策略,使其在流控制方面的应用前景广阔。许多流控制感兴趣的系统都具有对称性,如果忽略这些对称性,就会显著抑制朴素深度RL方法的学习和性能。使用由Kuramoto-Sivashinsky方程(KSE)组成的测试平台,等距致动器,以及最小化耗散和功耗成本的目标,我们证明了通过将深度RL问题移动到一个简化的空间,我们可以减轻深度RL天真应用中固有的局限性。我们证明了,与朴素的深度RL发现的策略相比,减少冗余的深度RL提高了数据效率,并提高了控制策略的有效性。有趣的是,由对称性感知控制代理学习的策略将系统推向强制KSE的平衡状态,该平衡状态通过延续连接到非强制KSE的平衡,尽管没有给出关于其存在的明确信息。即,为了实现其目标,RL算法发现并稳定系统的平衡状态。最后,我们证明了减少的控制策略是强大的观察和驱动信号噪声,以及系统参数,它没有观察到之前。

英文摘要:

Deep reinforcement learning (RL) is a data-driven, model-free method capable of discovering complex control strategies for macroscopic objectives in high-dimensional systems, making its application towards flow control promising. Many systems of flow control interest possess symmetries that, when neglected, can significantly inhibit the learning and performance of a naive deep RL approach. Using a test-bed consisting of the Kuramoto-Sivashinsky Equation (KSE), equally spaced actuators, and a goal of minimizing dissipation and power cost, we demonstrate that by moving the deep RL problem to a symmetry-reduced space, we can alleviate limitations inherent in the naive application of deep RL. We demonstrate that symmetry-reduced deep RL yields improved data efficiency as well as improved control policy efficacy compared to policies found by naive deep RL. Interestingly, the policy learned by the the symmetry aware control agent drives the system toward an equilibrium state of the forced KSE that is connected by continuation to an equilibrium of the unforced KSE, despite having been given no explicit information regarding its existence. I.e., to achieve its goal, the RL algorithm discovers and stabilizes an equilibrium state of the system. Finally, we demonstrate that the symmetry-reduced control policy is robust to observation and actuation signal noise, as well as to system parameters it has not observed before.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑