This paper focuses on the active flow control of a computational fluid dynamics simulation over a range of Reynolds numbers using deep reinforcement learning (DRL). More precisely, the proximal policy optimization (PPO) method is used to control the mass flow rate of four synthetic jets symmetrically located on the upper and lower sides of a cylinder immersed in a two-dimensional flow domain. The learning environment supports four flow configurations with Reynolds numbers 100, 200, 300 and 400, respectively. A new smoothing interpolation function is proposed to help the PPO algorithm to learn to set continuous actions, which is of great importance to effectively suppress problematic jumps in lift and allow a better convergence for the training process. It is shown that the DRL controller is able to significantly reduce the lift and drag fluctuations and to actively reduce the drag by approximately 5.7%, 21.6%, 32.7%, and 38.7%, at $Re$=100, 200, 300, and 400 respectively. More importantly, it can also effectively reduce drag for any previously unseen value of the Reynolds number between 60 and 400. This highlights the generalization ability of deep neural networks and is an important milestone to active flow control.
Deep artificial neural networks (ANNs) used together with deep reinforcement learning (DRL) are receiving growing attention due to their capabilities to control complex problems. This technique has been recently used to solve problems related to flow control. In this work, an ANN trained through a DRL agent is used to perform active flow control. Two-dimensional simulations of the flow around a cylinder are conducted and an active control based on two jets located on the walls of the cylinder is considered. By gathering information from the flow surrounding the cylinder, the ANN agent is able …
The present work investigates the application of Artificial Neural Networks (ANNs) to estimate the Reynolds ($Re$) number for flows around a cylinder. The data required to train the ANN was generated with our own implementation of a Lattice Boltzmann Method (LBM) code performing simulations of a 2-dimensional flow around a cylinder. As results of the simulations, we obtain the velocity field ($\vec{v}$) and the vorticity ($\vec{\nabla}\times\vec{v}$) of the fluid for 120 different values of $Re$ measured at different distances from the obstacle and use them to teach the ANN to predict the $Re$…
Università “Sapienza”（罗马第一大学）；University of Rome Tor Vergata（罗马第二大学）；Istituto dei Sistemi Complessi, CNR（意大利国家研究委员会复杂系统研究所）；The Abdus Salam International Centre for Theoretical Physics - ICTP（阿卜杜勒·萨拉姆国际理论物理中心）
We consider a model of two competing microswimming agents engaged in a pursue-evasion task within a low-Reynolds-number environment. Agents can only perform simple maneuvers and sense hydrodynamic disturbances, which provide ambiguous (partial) information about the opponent's position and motion. We frame the problem as a zero-sum game: The pursuer has to capture the evader in the shortest time, while the evader aims at deferring capture as long as possible. We show that the agents, trained via adversarial reinforcement learning, are able to overcome partial observability by discovering incre…
A local artificial neural network (LANN) framework is developed for turbulence modeling. The Reynolds-averaged Navier-Stokes (RANS) unclosed terms are reconstructed by artificial neural network (ANN) based on the local coordinate system which is orthogonal to the curved walls. We verify the proposed model for the flows over periodic hills. The correlation coefficients of the RANS unclosed terms predicted by the LANN model can be made larger than 0.96 in an a priori analysis, and the relative error of the unclosed terms can be made smaller than 18%. In an a posteriori analysis, detailed compari…
A modified Thickened Flame (TF) model based on Large Eddy Simulation methodology is used to investigate premixed combustion and the model predictions are evaluated by comparing with the piloted premixed stoichiometric methane-air flame data (Chen et al., 1996) for Reynolds numbers Re = 24,000 (flame F3) and Re=52,000 (flame F1). The basic idea of Thickened-Flame approach is that the flame front is artificially thickened to resolve on the computational LES grid while keeping the laminar flame speed ( ) constant. The artificially thickening of the flame front is obtained by enhancing the molecul…
Bead-based micro-swimmers are promising systems for payload delivery on the micro-scale. However, the principles underlying their optimal design are not yet fully understood. Here we study a simple device consisting of three arbitrarily-shaped beads connected by two springs. We analytically determine the most favorable kinematic parameters for sinusoidal driving, and show how the swimmer changes from being a pusher to a puller. For cargo carrying ellipsoidal beads, we perform geometric optimization under the constraint of a constant total volume or surface area, with the aim of maximizing the …
Conventionally, a microscopic particle that performs a reciprocal stroke cannot move through its environment. This is because at small scales, the response of simple Newtonian fluids is purely viscous and flows are time-reversible. We show that by contrast, fluid elasticity enables propulsion by reciprocal forcing that is otherwise impossible. We present experiments on rigid objects actuated reciprocally in viscous fluids, demonstrating for the first time a purely elastic propulsion set by the object's shape and boundary conditions. We describe two different artificial "swimmers" that experime…
Data-driven turbulence modeling has been considered an effective method for improving the prediction accuracy of Reynolds-averaged Navier-Stokes equations. Related studies aimed to solve the discrepancy of traditional turbulence modeling by acquiring specific patterns from high-fidelity data through machine learning methods, such as artificial neural networks. The present study focuses on the unsmoothness and prediction error problems from the aspect of feature selection and processing. The selection criteria for the input features are summarized, and an effective input set is constructed. The…
This paper focuses on a drag-reducing control strategy on a 2D-simulated laminar flow past a cylinder. Deep reinforcement learning algorithms have been implemented to discover efficient control schemes, using two synthetic jets located on the cylinder's poles as actuators and pressure sensors in the wake of the cylinder as feedback observation. The present work focuses on the efficiency and robustness of the identified control strategy and introduces a novel algorithm (S-PPO-CMA) to optimise the sensor layout. An energy-efficient control strategy reducing drag by 18.4% at Reynolds number 120 i…
In the limit of zero Reynolds number (Re), swimmers propel themselves exploiting a series of non-reciprocal body motions. For an artificial swimmer, a proper selection of the power source is required to drive its motion, in cooperation with its geometric and mechanical properties. Although various external fields (magnetic, acoustic, optical, etc.) have been introduced, electric fields are rarely utilized to actuate such swimmers experimentally in unbounded space. Here we use uniform and static electric fields to demonstrate locomotion of a bi-flagellated sphere at low Re via Quincke rotation.…
We present the first application of an Artificial Neural Network trained through a Deep Reinforcement Learning agent to perform active flow control. It is shown that, in a 2D simulation of the Karman vortex street at moderate Reynolds number (Re = 100), our Artificial Neural Network is able to learn an active control strategy from experimenting with the mass flow rates of two jets on the sides of a cylinder. By interacting with the unsteady wake, the Artificial Neural Network successfully stabilizes the vortex alley and reduces drag by about 8%. This is performed while using small mass flow ra…
Synthetic microswimmers show great promise in biomedical applications such as drug delivery and microsurgery. Their locomotion, however, is subject to stringent constraints due to the dominance of viscous over inertial forces at low Reynolds number (Re) in the microscopic world. Furthermore, locomotory gaits designed for one medium may become ineffective in a different medium. Successful biomedical applications of synthetic microswimmers rely on their ability to traverse biological environments with vastly different properties. Here we leverage the prowess of machine learning to present an alt…
We seek the best stroke sequences of a finite-size swimming predator chasing a non-motile point or finite--size prey at low Reynolds number. We use optimal control to seek the globally-optimal solutions for the former and RL for general situations. The predator is represented by a squirmer model that can translate forward and laterally, rotate and generate a stresslet flow. We identify the predator's best squirming sequences to achieve the time-optimal (TO) and efficiency-optimal (EO) predation. For a point prey, the TO squirmer executing translational motions favours a two-fold L-shaped traje…
We investigated the propagation of turbulent fronts in pipe flow at high Reynolds numbers by direct numerical simulation. We used a technique combining a moving frame of reference and an artificial damping to isolate the fronts in short periodic pipes, which enables us to explore the bulk Reynolds number up to Re = $10^5$ with affordable computation power. We measured the propagation speed of the downstream front and observed that a fit of $1.971-(Re/1925)^{-0.825}$ (in unit of bulk speed) well captures this speed above $Re\simeq 5000$. The speed increases monotonically as Re increases, in sta…
In this paper, we address a crucial point regarding the description of moderate to high Reynolds numbers aquatic swimmers. For decades, swimming animals have been classified in two different families of propulsive mechanisms based on the Reynolds number: the "resistive" swimmers, using local friction to produce the necessary thrust force for locomotion at low Reynolds number and the "reactive" swimmers, lying in the high Reynolds range, and using added mass acceleration (described by perfect fluid theory). However, inertial swimmers are also systems that dissipate energy, due to their finite s…
This work introduces a novel data-driven framework to formulate explicit algebraic Reynolds-averaged Navier-Stokes (RANS) turbulence closures. Recent years have witnessed a blossom in applying machine learning (ML) methods to revolutionize the paradigm of turbulence modeling. However, due to the black-box essence of most ML methods, it is currently hard to extract interpretable information and knowledge from data-driven models. To address this critical limitation, this work leverages deep learning with symbolic regression methods to discover hidden governing equations of Reynolds stress models…
A robust intermittency equation formulation for transition modeling in Spalart-Allmaras simulations of airfoil flows across a wide range of Reynolds numbers
We develop a wall model for large-eddy simulation (LES) that takes into account various pressure-gradient effects using multi-agent reinforcement learning (MARL). The model is trained using low-Reynolds-number flow over periodic hills with agents distributed on the wall along the computational grid points. The model utilizes a wall eddy-viscosity formulation as the boundary condition, which is shown to provide better predictions of the mean velocity field, rather than the typical wall-shear stress formulation. Each agent receives states based on local instantaneous flow quantities at an off-wa…
A realistic inflow boundary condition is essential for successful simulation of the developing turbulent boundary layer or channel flows. Recent advances in artificial intelligence (AI) have enabled the development of an inflow generator that performs better than the synthetic methods based on intuitions. In the present work, we applied generative adversarial networks (GANs), a representative of unsupervised learning, to generate an inlet boundary condition of turbulent channel flow. Upon learning the two-dimensional spatial structure of turbulence using data obtained from direct numerical sim…
High-lift Wing Separation Control via Bayesian Optimization and Deep Reinforcement Learning
TUAREG, Universitat Politècnica de Catalunya (UPC)（TUAREG，西班牙巴塞罗那理工大学）；Mechanical Engineering, Delft University of Technology (TU Delft)（代尔夫特理工大学机械工程系）；CASE, Barcelona Supercomputing Center (BSC)（巴塞罗那超级计算中心）
Machine learning has recently become a promising technique in fluid mechanics, especially for active flow control (AFC) applications. A recent work [J. Fluid Mech. (2019), vol. 865, pp. 281-302] has demonstrated the feasibility and effectiveness of deep reinforcement learning (DRL) in performing AFC over a circular cylinder at $Re = 100$, i.e., in the laminar flow regime. As a follow-up study, we investigate the same AFC problem at an intermediate Reynolds number, i.e., $Re = 1000$, where the turbulence in the flow poses great challenges to the control. The results show that the DRL agent can …
We investigate the role of the bulk eddy-viscosity variation on the well-known log-layer mismatch problem. An analysis of the mean momentum-balance shows that the modeled stress term close to the wall can dominate because of the bulk eddy-viscosity. Consequently, the momentum-balance equation lacks a degree-of-freedom and the mean velocity conforms to an incorrect profile to satisfy the momentum-balance. We show that zonal enforcement of the target mass flow-rate can be an effective strategy to introduce an additional degree of freedom to the mean momentum-balance, which led to a significant r…
Accounting for the Reynolds number is critical in numerical simulations of turbulence, particularly for subsonic flow. For Smoothed Particle Hydrodynamics (SPH) with constant artificial viscosity coefficient alpha, it is shown that the effective Reynolds number in the absence of explicit physical viscosity terms scales linearly with the Mach number - compared to mesh schemes, where the effective Reynolds number is largely independent of the flow velocity. As a result, SPH simulations with alpha=1 will have low Reynolds numbers in the subsonic regime compared to mesh codes, which may be insuffi…
We discuss artificial boundary conditions for stationary Navier-Stokes flows past bodies in the half-plane, for a range of low Reynolds numbers. When truncating the half-plane to a finite domain for numerical purposes, artificial boundaries appear. We present an explicit Dirichlet condition for the velocity at these boundaries in terms of an asymptotic expansion for the solution to the problem. We show a substantial increase in accuracy of the computed values for drag and lift when compared with results for traditional boundary conditions. We also analyze the qualitative behavior of the soluti…
Direct Numerical Simulations are used to solve turbulent flow and heat transfer over a variety of rough walls in a channel. The wall geometries are exactly resolved in the simulations. The aim is to understand the effect of roughness morphology and its scaling on the augmentation of heat transfer relative to that of skin friction. A number of realistic rough surface maps obtained from the scanning of gas turbine blades and internal combustion engines as well as several artificially generated rough surfaces are examined. In the first part of the paper, effects of statistical surface properties,…
In the study of microscopic flows, self-propulsion has been particularly topical in recent years, with the rise of miniature artificial swimmers as a new tool for flow control, low Reynolds number mixing, micromanipulation or even drug delivery. It is possible to take advantage of interfacial physics to propel these micro-robots, as demonstrated by recent experiments using the proximity of an interface, or the interface itself, to generate propulsion at low Reynolds number. This paper discusses how a nearby interface can provide the symmetry breaking necessary for propulsion. An overview of re…
Deep reinforcement learning (DRL) algorithms are rapidly making inroads into fluid mechanics, following the remarkable achievements of these techniques in a wide range of science and engineering applications. In this paper, a deep reinforcement learning (DRL) agent has been employed to train an artificial neural network (ANN) using computational fluid dynamics (CFD) data to perform active flow control (AFC) around a 2-D circular cylinder. Flow control strategies are investigated at a diameter-based Reynolds number Re_D = 100 using advantage actor-critic (A2C) algorithm by means of two symmetri…
Deep Reinforcement Learning in Action: Real-Time Control of Vortex-Induced Vibrations
Department of Mechanical Engineering, Tandon School of Engineering, New York, USA（机械工程系，工程学院，美国纽约）；Department of Mechanical Engineering, Delft University of Technology, Delft, Netherlands（机械工程系，代尔夫特理工大学，荷兰代尔夫特）
We computationally study the kinematics of a simple model reciprocal swimmer (asymmetric dumbbell) as a function of the Reynolds number (Re) and investigate how the onset and gradual increase of inertia impacts the swimming behavior: a reversal in the swim direction, flow directions, and the swim stroke. We divide the swim stroke into the expansion and compression of the two spheres and relate them to power and recovery strokes. We find that the switch in swim direction also corresponds to a switch in power and recovery strokes. We obtain expressions for the mean swimming velocity by collapsin…
We use the boundary element method to study the low-Reynolds number locomotion of a spherical model microorganism in a circular tube. The swimmer propels itself by tangen- tial or normal surface motion in a tube whose radius is on the order of the swimmer size. Hydrodynamic interactions with the tube walls significantly affect the average swimming speed and power consumption of the model microorganism. In the case of swimming parallel to the tube axis, the locomotion speed is always reduced (resp. increased) for swimmers with tangential (resp. normal) deformation. In all cases, the rate of wor…
Harvard University（哈佛大学）；California Institute of Technology（加州理工学院）；ETH Zurich（苏黎世联邦理工学院）
The predictive capabilities of turbulent flow simulations, critical for aerodynamic design and weather prediction, hinge on the choice of turbulence models. The abundance of data from experiments and simulations and the advent of machine learning have provided a boost to turbulence modeling efforts. However, simulations of turbulent flows remain hindered by the inability of heuristics and supervised learning to model the near-wall dynamics. We address this challenge by introducing scientific multi-agent reinforcement learning (SciMARL) for the discovery of wall models for large-eddy simulation…
Navigation of a Three-Link Microswimmer via Deep Reinforcement Learning
Department of Mechanics and Engineering Science at College of Engineering（机械与工程科学系）；Department of Mechanical Engineering（机械工程系）；Santa Clara University（圣克拉拉大学）；Department of Applied Mathematics（应用数学系）
The motivation of this study is to leverage recent breakthroughs in artificial intelligence research to unlock novel solutions to important scientific problems encountered in computational science. To address the human intelligence limitations in discovering reduced-order models, we propose to supplement human thinking with artificial intelligence. Our three-pronged strategy consists of learning (i) models expressed in analytical form, (ii) which are evaluated a posteriori, and iii) using exclusively integral quantities from the reference solution as prior knowledge. In point (i), we pursue in…
Zhejiang University（浙江大学）；Ningbo Institute of Technology, Beihang University（北京航空航天大学宁波研究院）
By imitating biological microswimmers, microrobots can be designed to accomplish targeted delivery of cargos and biomedical manipulations at microscale. However, it is still a great challenge to enable microrobots to maneuver in a complex environment. Machine learning algorithms offer a tool to boost mobility and flexibility of a synthetic microswimmer, hence could help us design truly smart microrobots. In this work, we investigate how a model of sea urchin sperm cell can self-learn chemotactic motion in a chemoattractant concentration field. We employ an artificial neural network to act as a…
