arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2302.05081physics.flu-dyncond-mat.soft
收藏

通过强化学习控制流体流动中的波动微游泳者

Steering undulatory micro-swimmers in a fluid flow through reinforcement learning

Zakarya El Khiyati, Raphael Chesneaux, Laetitia Giraldi, Jeremie Bec

摘要:

这项工作的目的是寻找最佳的导航策略薄,可变形的microswimmer,在粘性流体中传播的正弦波动沿其细长的身体。这些活跃的细丝被嵌入在一个规定的,非均匀的流动,其中他们的游泳波动必须与漂移,应变和变形所造成的外部速度场竞争。游泳和导航紧密结合的这种复杂情况可以使用各种强化学习方法来解决。每个游泳者只能访问关于其配置的受限信息,并且必须相应地在有限集合中选择动作。然后,优化问题在于找到在给定方向上导致最有效位移的策略。据发现,通常的方法不收敛,这个陷阱被解释为一个综合的结果的非马尔可夫的决策过程中,连同高度混乱的性质的动态,这是负责学习效率的高变异性。尽管如此,我们还是提供了一种构建有效策略的替代方法,该方法基于运行几个独立的Q学习实现。这使得一组可接受的政策,其属性可以详细研究和比较,以评估其效率和鲁棒性的建设。

英文摘要:

This work aims at finding optimal navigation policies for thin, deformable microswimmers that progress in a viscous fluid by propagating a sinusoidal undulation along their slender body. These active filaments are embedded in a prescribed, non-homogeneous flow, in which their swimming undulations have to compete with the drifts, strains, and deformations inflicted by the outer velocity field. Such an intricate situation, where swimming and navigation are tightly bonded, is addressed using various methods of reinforcement learning. Each swimmer has only access to restricted information on its configuration and has to select accordingly an action among a limited set. The optimisation problem then consists in finding the policy leading to the most efficient displacement in a given direction. It is found that usual methods do not converge and this pitfall is interpreted as a combined consequence of the non-Markovianity of the decision process, together with the highly chaotic nature of the dynamics, which is responsible for high variability in learning efficiencies. Still, we provide an alternative method to construct efficient policies, which is based on running several independent realisations of Q-learning. This allows the construction of a set of admissible policies whose properties can be studied in detail and compared to assess their efficiency and robustness.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑