OpenSBLI is an open-source code-generation system for compressible fluid dynamics (CFD) on heterogeneous computing architectures. Written in Python, OpenSBLI is an explicit high-order finite-difference solver on structured curvilinear meshes. Shock-capturing is performed by a choice of high-order Weighted Essentially Non-Oscillatory (WENO) or Targeted Essentially Non-Oscillatory (TENO) schemes. OpenSBLI generates a complete CFD solver in the Oxford Parallel Structured (OPS) domain specific language. The OPS library is embedded in C code, enabling massively-parallel execution of the code on a variety of high-performance-computing architectures, including GPUs. The present paper presents a code base that has been completely rewritten from the earlier proof of concept (Jacobs et al, JoCS 18 (2017), 12-23), allowing shock capturing, coordinate transformations for complex geometries, and a wide range of boundary conditions, including solid walls with and without heat transfer. A suite of validation and verification cases are presented, plus demonstration of a large-scale Direct Numerical Simulation (DNS) of a transitional Shockwave Boundary Layer Interaction (SBLI). The code is shown to have good weak and strong scaling on multi-GPU clusters. We demonstrate that code-generation and domain specific languages are suitable for performing efficient large-scale simulations of complex fluid flows on emerging computing architectures.
The speed of sound in two-phase pipe flow systems is often several orders of magnitude greater than the travelling speed of hydraulic information (volume fractions.) Dynamically simulating such flows requires resolution of acoustic and hydraulic waves existing at vastly different spatial and temporal scales. If simulated on the same numerical grid, the need for accuracy in hydraulic waves will necessitate an exaggerated resolution of acoustic waves. Likewise, time steps restricted by the speed of sound are small compared to the time scales active in hydraulic waves. This constitutes a waste of…
A low-dissipation numerical method based on boundary variation diminishing principle for compressible gas-liquid two-phase flows with phase change on unstructured grid
We propose a constraint preserving discontinuous Galerkin method for ideal compressible MHD in two dimensions and using Cartesian grids, which automatically maintains the global divergence-free property. The approximation of the magnetic field is achieved using Raviart-Thomas polynomials and the DG scheme is based on evolving certain moments of these polynomials which automatically guarantees divergence-free property. We also develop HLL-type multi-dimensional Riemann solvers to estimate the electric field at vertices which are consistent with the 1-D Riemann solvers. When limiters are used, t…
We present a dimensionally split method for computing solutions to the compressible Navier-Stokes equations on Cartesian cut cell meshes. The method is globally second order accurate in the L1 norm, fully conservative, and allows the use of time steps determined by the regular grid spacing. We provide a description of the three-dimensional implementation of the method and evaluate its numerical performance by computing solutions to a number of test problems ranging from the nearly incompressible to the highly compressible flow regimes. All the computed results show good agreement with referenc…
This study aims to identify the properties, advantages, and drawbacks of some common (and some less common) sub-grid scale (SGS) models for large eddy simulation of low Mach compressible flows using high order spectral elements. The models investigated are the classical constant coefficient Smagorinsky-Lilly, the model by Vreman and two variants of a dynamic SGS (DSGS) model designed to stabilize finite and spectral elements for transport dominated problems. In particular, we compare one variant of DSGS that is based on a time-dependent residual version (R-DSGS) in contrast to a time-independe…
GPU-based compressible lattice Boltzmann simulations on non-uniform grids using standard C++ parallelism: From best practices to aerodynamics, aeroacoustics and supersonic flow simulations
Institute for Advanced Study, Beijing Normal - Hong Kong Baptist University（北京师范大学-香港浸会大学联合国际学院先进研究院）；Computer Science Department, University of Geneva（日内瓦大学计算机系）
High-Resolution Weighted Essentially Non-Oscillatory Compact Least-Squares Schemes with Implicit Time Integration for Compressible Navier-Stokes Equations on Curvilinear Grids
Ningbo University（宁波大学）；Northeast Normal University（东北师范大学）；Tsinghua University（清华大学）
We describe the development of a flux-limited gray radiation solver for the compressible astrophysics code, CASTRO. CASTRO uses an Eulerian grid with block-structured adaptive mesh refinement based on a nested hierarchy of logically-rectangular variable-sized grids with simultaneous refinement in both space and time. The gray radiation solver is based on a mixed-frame formulation of radiation hydrodynamics. In our approach, the system is split into two parts, one part that couples the radiation and fluid in a hyperbolic subsystem, and another parabolic part that evolves radiation diffusion and…
The evolution of any complex dynamical system is described by its state derivative operators. However, the extraction of the exact N-order state derivative operators is often inaccurate and requires approximations. The open-source CFD code called BROADCAST discretises the compressible Navier-Stokes equations and then extracts the linearised Nderivative operators through Algorithmic Differentiation (AD) providing a toolbox for laminar flow dynamic analyses. Furthermore, the gradients through adjoint derivation are extracted either by transposition of the linearised operator or through the backw…
We present a numerical formulation for the solution of non-isothermal, compressible, Navier-Stokes equations with thermal fluctuations to describe mesoscale transport phenomena in multispecies fluid mixtures. The novelty of our numerical method is the use of staggered grid momenta along with a finite volume discretization of the thermodynamic variables to solve the resulting stochastic partial differential equations. The key advantages of the numerical scheme are significantly simplified and compact discretization of the diffusive and stochastic momentum fluxes, and an unambiguous prescription…
A numerical procedure was developed for solving equations for compressible granular multiphase flows in which the particle volume fraction can range dynamically from very dilute to very dense. The procedure uses a low-dissipation and high-order numerical method that can describe shocks and incorporates a particulate model based on kinetic theory. The algorithm separates edges of a computational cell into gas and solid sections where gas- and granular-phase Riemann problems are solved independently. Solutions from these individual Riemann problems are combined to assemble the fully coupled conv…
This work describes three diffuse-interface methods for the simulation of immiscible, compressible multiphase fluid flows and elastic-plastic deformation in solids. The first method is the localized-artificial-diffusivity approach of Cook (2007), Subramaniam et al., (2018), and Adler and Lele (2019), in which artificial diffusion terms are added to the conservation equations. The second method is the gradient-form approach that is based on the quasi-conservative method of Shukla et al., (2010) and Tiwari et al., (2013), in which the diffusion and sharpening terms (together called regularizatio…
A numerical approach for solving evolutionary partial differential equations in two and three space dimensions on block-based adaptive grids is presented. The numerical discretization is based on high-order, central finite-differences and explicit time integration. Grid refinement and coarsening are triggered by multiresolution analysis, i.e. thresholding of wavelet coefficients, which allow controlling the precision of the adaptive approximation of the solution with respect to uniform grid computations. The implementation of the scheme is fully parallel using MPI with a hybrid data structure.…
In this note, the importance of spectral properties of viscous flux discretization in solving compressible Navier-Stokes equations for turbulent flow simulations is discussed. We studied six different methods, divided into two different classes, with poor and better representation of spectral properties at high wavenumbers. Both theoretical and numerical results have revealed that the method with better properties at high wavenumbers, denoted as $α$-damping type discretization, produced superior solutions compared to the other class of methods. The proposed compact $α$-damping method converged…
This paper investigates the application of mesh adaptation techniques in the Non-Ideal Compressible Fluid Dynamic (NICFD) regime, a region near the vapor-liquid saturation curve where the flow behavior significantly departs from the ideal gas model, as indicated by a value of the fundamental derivative of gasdynamics less than one. A recent interpolation-free finite-volume adaptive scheme is exploited to modify the grid connectivity in a conservative way, and the governing equations for compressible inviscid flows are solved within the Arbitrary Lagrangian Eulerian framework by including speci…
This paper presents the benchmarking and scaling studies of a GPU accelerated three dimensional compressible magnetohydrodynamic code. The code is developed keeping an eye to explain the large and intermediate scale magnetic field generation is cosmos as well as in nuclear fusion reactors in the light of the theory given by Eugene Newman Parker. The spatial derivatives of the code are pseudo-spectral method based and the time solvers are explicit. GPU acceleration is achieved with minimal code changes through OpenACC parallelization and use of NVIDIA CUDA Fast Fourier Transform library (cuFFT)…
The present paper addresses the development and implementation of the first high-order Flux Reconstruction (FR) solver for high-speed flows within the open-source COOLFluiD (Computational Object-Oriented Libraries for Fluid Dynamics) platform. The resulting solver is fully implicit and able to simulate compressible flow problems governed by either the Euler or the Navier-Stokes equations in two and three dimensions. Furthermore, it can run in parallel on multiple CPU-cores and is designed to handle unstructured grids consisting of both straight and curved edged quadrilateral or hexahedral elem…
Affordable, high order simulations of turbulent flows on unstructured grids for very high Reynolds number flows require wall models for efficiency. However, different wall models have different accuracy and stability properties. Here, we develop a kinetic energy stability estimate to investigate stability of wall model boundary conditions. Using this norm, two wall models are studied, a popular equilibrium stress wall model, which is found to be unstable and the dynamic slip wall model which is found to be stable. These results are extended to the discrete case using the Summation by parts (SB…
An overset grid method was used to investigate the interaction between a particle-laden flow and a circular cylinder. The overset grid method was implemented in the Pencil Code , a high-order finite-difference code for compressible flow simulation. High-order summation-by-part operators were used at the cylinder boundary, and both bi-linear Lagrangian and bi-quadratic spline interpolation was used to communicate between the background grid and the body-conformal cylindrical grid. The performance of the overset grid method was assessed to benchmark cases of steady and unsteady flows past a cyli…
We focus on implementing and optimizing a sixth-order finite-difference solver for simulating compressible fluids on a GPU using third-order Runge-Kutta integration. Since graphics processing units perform well in data-parallel tasks, this makes them an attractive platform for fluid simulation. However, high-order stencil computation is memory-intensive with respect to both main memory and the caches of the GPU. We present two approaches for simulating compressible fluids using 55-point and 19-point stencils. We seek to reduce the requirements for memory bandwidth and cache size in our methods…
Typical multispecies compressible Navier-Stokes computations employ conservative equations for mass fraction transport. Upwind discretisations of these governing equations produce spurious pressure oscillations at diffuse contact surfaces between gases of differing ratio of specific heat capacities which degrade the convergence rate of the algorithm. Adding quasi-conservative equations for volume fraction can solve this error, however this approach has been derived only for immiscible fluids. Here, a five-equation quasi-conservative model is proposed that includes the effects of species diffus…
This report summarizes the major progresses to develop the dynamic core for next-generation atmospherical model for both numerical weather prediction and climate simulation. The numerical framework is based on a general formulation, so-called multi-moment constrained finite volume (MCV) method, which is well-balanced among solution quality (accuracy and robustness), algorithmic simplicity, computational efficiency and flexibility for model configuration. A local high-order limiting projection is also devised to remove spurious oscillations and noises in numerical solutions, which allows the nu…
A two-phase model and its application to wavefields numerical simulation are discussed in the context of modeling of compressible fluid flows in elastic porous media. The derivation of the model is based on a theory of thermodynamically compatible systems and on a model of nonlinear elastoplasticity combined with a two-phase compressible fluid flow model. The governing equations of the model include phase mass conservation laws, a total momentum conservation law, an equation for the relative velocities of the phases, an equation for mixture distortion, and a balance equation for porosity. They…
This paper presents the first high-order computational fluid dynamics (CFD) simulations of static and spinning golf balls at realistic flow conditions. The present results are shown to capture the complex fluid dynamics inside the dimples which lead to drag reduction versus a smooth sphere, and compare well to previous experimental and computational studies. The high--order Flux Reconstruction method has been paired with the Artificial Boundary overset method to enable simplified mesh generation and grid motion. The compressible Navier--Stokes equations are modeled using a scale--resolving Lar…
The main result in this paper is a provably entropy stable shock capturing approach for the high order entropy stable DGSEM based on a hybrid blending with a subcell low order variant. Since it is possible to rewrite a high order SBP operator into an equivalent conservative finite volume form, we were able to design a low order scheme directly with the LGL nodes that is compatible to the discrete entropy analysis used for the proof of the entropy stable DGSEM. Furthermore, we present a hybrid low order/high order discretisation where it is possible to seamlessly blend between the two approache…
High-order numerical methods for unstructured grids combine the superior accuracy of high-order spectral or finite difference methods with the geometric flexibility of low-order finite volume or finite element schemes. The Flux Reconstruction (FR) approach unifies various high-order schemes for unstructured grids within a single framework. Additionally, the FR approach exhibits a significant degree of element locality, and is thus able to run efficiently on modern streaming architectures, such as Graphical Processing Units (GPUs). The aforementioned properties of FR mean it offers a promising …
