Parallelization schemes are essential in order to exploit the full benefits of multi-core architectures. In said architectures, the most comprehensive parallelization API is OpenMP. However, the introduction of correct and optimal OpenMP parallelization to applications is not always a simple task, due to common parallel management pitfalls, architecture heterogeneity and the current necessity for human expertise in order to comprehend many fine details and abstract correlations. To ease this process, many automatic parallelization compilers were created over the last decade. Harel et al. [2020] tested several source-to-source compilers and concluded that each has its advantages and disadvantages and no compiler is superior to all other compilers in all tests. This indicates that a fusion of the compilers' best outputs under the best hyper-parameters for the current hardware setups can yield greater speedups. To create such a fusion, one should execute a computationally intensive hyper-parameter sweep, in which the performance of each option is estimated and the best option is chosen. We created a novel parallelization source-to-source multi-compiler named ComPar, which uses code segmentation-and-fusion with hyper-parameters tuning to achieve the best parallel code possible without any human intervention while maintaining the program's validity. In this paper we present ComPar and analyze its results on NAS and PolyBench benchmarks. We conclude that although the resources ComPar requires to produce parallel code are greater than other source-to-source parallelization compilers - as it depends on the number of parameters the user wishes to consider, and their combinations - ComPar achieves superior performance overall compared to the serial code version and other tested parallelization compilers. ComPar is publicly available at: https://github.com/Scientific-Computing-Lab-NRCN/compar.
Multicore has emerged as a typical architecture model since its advent and stands now as a standard. The trend is to increase the number of cores and improve the performance of the memory system. Providing an efficient multicore implementation for a important algorithmic kernel is a genuine contribution. From a methodology standpoint, this should be done at the level of the underlying paradigm if any. In this paper, we study the cases of {\em dynamic programming} and {\em greedy algorithms}, which are two major algorithmic paradigms. We exclusively consider directives-based loop parallelizatio…
OpenMP provides programmers with directives to parallelize DOALL loops such as parallel for and, more recently, taskloop for task-based parallelism. On the other hand, when it is possible to prove that a loop is DOACROSS, programmers can try to parallelize it through parallel for and to use the OpenMP ordered directive to mark the region of the loop that has to be executed sequentially. However, when neither of the previous two cases can be proven, programmers have to be conservative and assume that the loop is DOACROSS (actually may DOACROSS). Previous work proposed speculative support for ta…
Advising OpenMP Parallelization via a Graph-Based Approach with Transformers
Ben-Gurion University of the Negev（内盖夫本-古里安大学）；Israel Atomic Energy Commission（以色列原子能委员会）；Intel Labs（英特尔实验室）；Nuclear Research Center – Negev（内盖夫核研究中心）；Technion – Israel Institute of Technology（以色列理工学院）
We present our experience with the modernization on the GR-MHD code BHAC, aimed at improving its novel hybrid (MPI+OpenMP) parallelization scheme. In doing so, we showcase the use of performance profiling tools usable on x86 (Intel-based) architectures. Our performance characterization and threading analysis provided guidance in improving the concurrency and thus the efficiency of the OpenMP parallel regions. We assess scaling and communication patterns in order to identify and alleviate MPI bottlenecks, with both runtime switches and precise code interventions. The performance of optimized ve…
Ben-Gurion University of the Negev（内盖夫本-古里安大学）；Nuclear Research Center – Negev（内盖夫核研究中心）；Technion – Israel Institute of Technology（以色列理工学院）
In past years, the world has switched to many-core and multi-core shared memory architectures. As a result, there is a growing need to utilize these architectures by introducing shared memory parallelization schemes to software applications. OpenMP is the most comprehensive API that implements such schemes, characterized by a readable interface. Nevertheless, introducing OpenMP into code is challenging due to pervasive pitfalls in management of parallel shared memory. To facilitate the performance of this task, many source-to-source (S2S) compilers have been created over the years, tasked with…
Currently, multi/many-core CPUs are considered standard in most types of computers including, mobile phones, PCs or supercomputers. However, the parallelization of applications as well as refactoring/design of applications for efficient hardware usage remains restricted to experts who have advanced technical knowledge and who can invest time tuning their software. In this context, the compilation community has proposed different methods for automatic parallelization, but their focus is traditionally on loops and nested loops with the support of polyhedral techniques. In this study, we propose …
We use OpenMP to target hardware accelerators (GPUs) on Summit, a newly deployed supercomputer at the Oak Ridge Leadership Computing Facility (OLCF), demonstrating simplified access to GPU devices for users of our astrophysics code GenASiS and useful speedup on a sample fluid dynamics problem. We modify our workhorse class for data storage to include members and methods that significantly streamline the persistent allocation of and association to GPU memory. Users offload computational kernels with OpenMP target directives that are rather similar to constructs already familiar from multi-core …
Parallelization of JOREK-STARWALL for non-linear MHD simulations including resistive walls (Report of the EUROfusion High Level Support Team Projects JORSTAR/JORSTAR2)
托卡马克装置内的大尺度等离子体不稳定性会受到传导容器壁中流动的电流的影响。这涉及到非线性等离子体动力学及其与壁电流的相互作用。为了研究这个问题，求解磁流体动力学（MHD）方程的代码，称为JOREK [Huysmans G.T.A.和Czarny O. NF 47，659（2007）; Czarny O.和Huysmans G. JCP 227，7423（2008）]，偶联[Hoelzl M.，Journal of Physics：Conference Series，401，012010（2012）]，其具有用于真空区域的模型和名为STARWALL的电阻性导电结构[Merkel P.，Strumberger E.，arXiv：150804911（2015）]。JOREK-STARWALL模型已经应用于执行垂直位移事件（VDE）、电阻壁模式（RWM）、静态H模式和垂直反冲ELM触发的模拟。在项目开始时，由于STARWALL和JOREK中相应的耦合例程消耗大量内存和挂钟时间，无法解决具有大量有限元三角形的实际墙壁结构。此外，STARWALL代码和JOREK耦合例程都仅通过OpenMP部分并行化。该项目的目的是实现MPI并行化，以减少内存消耗和执行时间，使大分辨率的模拟成为可能。
One of the barriers to the adoption of parallel computing is the inherent complexity of its programming. The Open Multi-Processing (OpenMP) Application Programming Interface (API) facilitates such implementations, providing high abstraction level directives. On another front, new architectures aimed at low energy consumption have been developed, such as the Greenwaves Technologies GAP8, a Multi-Processor System-on-Chip (MPSoC) based on the Parallel Ultra Low Power (PULP) Platform. The GAP8 has an 8-core cluster and a Fabric Controller(FC) master core. Parallel programming with GAP8 is very pro…
Asynchronous Many-task (AMT) runtime systems have gained increasing acceptance in the HPC community due to the performance improvements offered by fine-grained tasking runtime systems. At the same time, C++ standardization efforts are focused on creating higher-level interfaces able to replace OpenMP or OpenACC in modern C++ codes. These higher level functions have been adopted in standards conforming runtime systems such as HPX, giving users the ability to simply utilize fork-join parallelism in their own codes. Despite innovations in runtime systems and standardization efforts users face eno…
Evaluating OpenMP Offloading for Intra-node Multi-GPU Programming across NVIDIA, AMD, and Intel Architectures: A 3D Heat Transfer Case Study
Rudolfovo-Science and Technology Centre Novo mesto（鲁多尔沃夫诺沃梅斯托科学与技术中心）；Faculty of Mechanical Engineering, University of Ljubljana（卢布尔雅那大学机械工程学院）；Department of Computer Science, FSTM, University of Luxembourg（卢森堡大学科学技术学院计算机科学系）
The Gaia Astrometric Verification Unit-Global Sphere Reconstruction (AVU-GSR) Parallel Solver aims to find the astrometric parameters for $\sim$10$^8$ stars in the Milky Way, the attitude and the instrumental specifications of the Gaia satellite, and the global parameter $γ$ of the post Newtonian formalism. The code iteratively solves a system of linear equations, $\mathbf{A} \times \vec{x} = \vec{b}$, where the coefficient matrix $\mathbf{A}$ is large ($\sim$$10^{11} \times 10^8$ elements) and sparse. To solve this system of equations, the code exploits a hybrid implementation of the iterativ…
Deep neural networks (DNNs) have emerged as successful solutions for variety of artificial intelligence applications, but their very large and deep models impose high computational requirements during training. Multi-GPU parallelization is a popular option to accelerate demanding computations in DNN training, but most state-of-the-art multi-GPU deep learning frameworks not only require users to have an in-depth understanding of the implementation of the frameworks themselves, but also apply parallelization in a straight-forward way without optimizing GPU utilization. In this work, we propose a…
