Deep reinforcement learning (DRL) has recently been adopted in a wide range of physics and engineering domains for its ability to solve decision-making problems that were previously out of reach due to a combination of non-linearity and high dimensionality. In the last few years, it has spread in the field of computational mechanics, and particularly in fluid dynamics, with recent applications in flow control and shape optimization. In this work, we conduct a detailed review of existing DRL applications to fluid mechanics problems. In addition, we present recent results that further illustrate the potential of DRL in Fluid Mechanics. The coupling methods used in each case are covered, detailing their advantages and limitations. Our review also focuses on the comparison with classical methods for optimal control and optimization. Finally, several test cases are described that illustrate recent progress made in this field. The goal of this publication is to provide an understanding of DRL capabilities along with state-of-the-art applications in fluid dynamics to researchers wishing to address new problems with these methods.
Meshing is a critical, but user-intensive process necessary for stable and accurate simulations in computational fluid dynamics (CFD). Mesh generation is often a bottleneck in CFD pipelines. Adaptive meshing techniques allow the mesh to be updated automatically to produce an accurate solution for the problem at hand. Existing classical techniques for adaptive meshing require either additional functionality out of solvers, many training simulations, or both. Current machine learning techniques often require substantial computational cost for training data generation, and are restricted in scope…
LinguaFluid: Language Guided Fluid Control via Semantic Rewards in Reinforcement Learning
College of Environmental and Resource Sciences, Zhejiang University, Hangzhou, China（浙江大学环境与资源学院）；School of Engineering, Westlake University, Hangzhou, China（西湖大学工程学院）
Bielefeld University（比勒费尔德大学）；Leibniz Institute for Neurobiology（莱布尼茨神经生物学研究所）；Otto-von-Guericke University（奥托·冯·格里克大学）
Recent developments in machine-learning algorithms have led to impressive performance increases in many traditional application scenarios of artificial intelligence research. In the area of deep reinforcement learning, deep learning functional architectures are combined with incremental learning schemes for sequential tasks that include interaction-based, but often delayed feedback. Despite their impressive successes, modern machine-learning approaches, including deep reinforcement learning, still perform weakly when compared to flexibly adaptive biological systems in certain naturally occurri…
Mila – Quebec Artificial Intelligence Institute（魁北克人工智能研究所）；McGill University（麦吉尔大学）；Ottawa Hospital Research Institute（渥太华医院研究所在）；University of Ottawa（渥太华大学）
Plug-and-Play Benchmarking of Reinforcement Learning Algorithms for Large-Scale Flow Control
TU Dortmund University（图卢兹大学）；Lamarr Institute for Machine Learning（拉马尔机器学习研究所）；Technical University Munich（慕尼黑技术大学）；Munich Center for Machine Learning（慕尼黑机器学习中心）
Mechanical ventilation is a key form of life support for patients with pulmonary impairment. Healthcare workers are required to continuously adjust ventilator settings for each patient, a challenging and time consuming task. Hence, it would be beneficial to develop an automated decision support tool to optimize ventilation treatment. We present DeepVent, a Conservative Q-Learning (CQL) based offline Deep Reinforcement Learning (DRL) agent that learns to predict the optimal ventilator parameters for a patient to promote 90 day survival. We design a clinically relevant intermediate reward that e…
Despite the low dimensionalities of dissipative viscous fluids, reinforcement learning (RL) requires many observables in fluid control problems. This is because the observables are assumed to follow a policy-independent Markov decision process in the RL framework. By including policy parameters as arguments of a value function, we construct a consistent algorithm with partially observable condition. Using typical examples of active flow control, we show that our algorithm is more stable and efficient than the existing RL algorithms, even under a small number of observables.
In recent years, Artificial Neural Networks (ANNs) and Deep Learning have become increasingly popular across a wide range of scientific and technical fields, including Fluid Mechanics. While it will take time to fully grasp the potentialities as well as the limitations of these methods, evidence is starting to accumulate that point to their potential in helping solve problems for which no theoretically optimal solution method is known. This is particularly true in Fluid Mechanics, where problems involving optimal control and optimal design are involved. Indeed, such problems are famously diffi…
In the past couple of years, the interest of the fluid mechanics community for deep reinforcement learning (DRL) techniques has increased at fast pace, leading to a growing bibliography on the topic. While the capabilities of DRL to solve complex decision-making problems make it a valuable tool for active flow control, recent publications also demonstrated applications to other fields, such as shape optimization or microfluidics. The present work aims at proposing an exhaustive review of the existing literature, and is a follow-up to our previous review on the topic. The contributions are regr…
Discrete mechanics makes it possible to formulate any problem of fluid mechanics or fluid-structure interaction in velocity and potentials of acceleration; the equation system consists of a single vector equation and potentials updates. The scalar potential of the acceleration represents the pressure stress and the vector potential is related to the rotational-shear stress. The formulation of the equation of motion can be expressed in the form of a splitting which leads to an exact projection method; the application of the divergence operator to the discrete motion equation exhibits, without a…
A formulation of the boundary integral method for solving partial differential equations has been developed whereby the usual weakly singular integral and the Cauchy principal value integral can be removed analytically. The broad applicability of the approach is illustrated with a number of problems of practical interest to fluid and continuum mechanics including the solution of the Laplace equation for potential flow, the Helmholtz equation as well as the equations for Stokes flow and linear elasticity.
In conventional fluid mechanics, the chemical composition and thermodynamic state of a fluid-solid interface are not considered when establishing velocity-field boundary conditions. As a consequence, fluid simulations are usually not able to generate different outputs when interfacial materials are varied. By considering an atomistic description of matter, theoretical determination of material-specific boundary conditions becomes possible, thereby providing an improved alternative to the completely-invariant no-slip condition. Such a scheme constitutes a multiscale approach to fluid dynamics i…
Particle shifting techniques (PST) have been used to improve the accuracy of the Smoothed Particle Hydrodynamics (SPH) method. Shifting ensures that the particles are distributed homogeneously in space. This may be performed by moving the particles using a transport velocity. In this paper, we propose an extension to the class of Transport Velocity Formulation (TVF) methods. We derive the equations in a consistent manner and show that there are additional terms that significantly improve the accuracy of the method. In particular, we apply this to the Entropically Damped Artificial Compressibil…
We present approaches for the study of fluid-structure interactions subject to thermal fluctuations. A mixed mechanical description is utilized combining Eulerian and Lagrangian reference frames. We establish general conditions for operators coupling these descriptions. Stochastic driving fields for the formalism are derived using principles from statistical mechanics. The stochastic differential equations of the formalism are found to exhibit significant stiffness in some physical regimes. To cope with this issue, we derive reduced stochastic differential equations for several physical regime…
An open-source finite volume toolbox for solid mechanics and fluid-solid interaction simulations
University College Dublin（都柏林大学）；University of Zenica（泽尼察大学）；NV Bekaert SA（Bekaert公司）；University of Zagreb（萨格勒布大学）；Johannes Kepler University（约翰内斯·开普勒大学）
