This work aims at finding optimal navigation policies for thin, deformable microswimmers that progress in a viscous fluid by propagating a sinusoidal undulation along their slender body. These active filaments are embedded in a prescribed, non-homogeneous flow, in which their swimming undulations have to compete with the drifts, strains, and deformations inflicted by the outer velocity field. Such an intricate situation, where swimming and navigation are tightly bonded, is addressed using various methods of reinforcement learning. Each swimmer has only access to restricted information on its configuration and has to select accordingly an action among a limited set. The optimisation problem then consists in finding the policy leading to the most efficient displacement in a given direction. It is found that usual methods do not converge and this pitfall is interpreted as a combined consequence of the non-Markovianity of the decision process, together with the highly chaotic nature of the dynamics, which is responsible for high variability in learning efficiencies. Still, we provide an alternative method to construct efficient policies, which is based on running several independent realisations of Q-learning. This allows the construction of a set of admissible policies whose properties can be studied in detail and compared to assess their efficiency and robustness.
Optimum control strategies for maximum thrust production in underwater undulatory swimming
Laboratoire de Tribologie et Dynamique des Systemes, École Centrale de Lyon(里昂中央理工学院);Université Côte d’Azur, CNRS, INPHYNI(蔚蓝海岸大学);Departamento de Física Fundamental, Universidad Nacional de Educación a Distancia(西班牙国立远程教育大学);Université Côte d’Azur, CNRS, I3S(蔚蓝海岸大学);IUF, Paris(法国高等研究学院)
Fish-inspired tracking of underwater turbulent plumes
Graduate Aerospace Laboratories, California Institute of Technology(加州理工学院研究生航空实验室);Mechanical and Civil Engineering, California Institute of Technology(加州理工学院机械与土木工程系)