Multicore has emerged as a typical architecture model since its advent and stands now as a standard. The trend is to increase the number of cores and improve the performance of the memory system. Providing an efficient multicore implementation for a important algorithmic kernel is a genuine contribution. From a methodology standpoint, this should be done at the level of the underlying paradigm if any. In this paper, we study the cases of {\em dynamic programming} and {\em greedy algorithms}, which are two major algorithmic paradigms. We exclusively consider directives-based loop parallelization through OpenMP and investigate necessary pre-transformations to reach a regular parallel form. We evaluate our methodology with a selection of well-known combinatorial optimization problems on an INTEL Broadwell processor. Key points for scalability are discussed before and after experimental results. Our immediate perspective is to extend our study to the manycore case, with a special focus on NUMA configurations.
Modern OpenMP threading techniques are used to convert the MPI-only Hartree-Fock code in the GAMESS program to a hybrid MPI/OpenMP algorithm. Two separate implementations that differ by the sharing or replication of key data structures among threads are considered, density and Fock matrices. All implementations are benchmarked on a super-computer of 3,000 Intel Xeon Phi processors. With 64 cores per processor, scaling numbers are reported on up to 192,000 cores. The hybrid MPI/OpenMP implementation reduces the memory footprint by approximately 200 times compared to the legacy code. The MPI/Ope…
OpenMP is a cross-platform API that extends C, C++ and Fortran and provides shared-memory parallelism platform for those languages. The use of many cores and HPC technologies for scientific computing has been spread since the 1990s, and now takes part in many fields of research. The relative ease of implementing OpenMP, along with the development of multi-core shared memory processors (such as Intel Xeon Phi) makes OpenMP a favorable method for parallelization in the process of modernizing a legacy codes. Legacy scientific codes are usually holding large number of physical arrays which being u…
Parallelization schemes are essential in order to exploit the full benefits of multi-core architectures. In said architectures, the most comprehensive parallelization API is OpenMP. However, the introduction of correct and optimal OpenMP parallelization to applications is not always a simple task, due to common parallel management pitfalls, architecture heterogeneity and the current necessity for human expertise in order to comprehend many fine details and abstract correlations. To ease this process, many automatic parallelization compilers were created over the last decade. Harel et al. [2020…
OpenMP provides programmers with directives to parallelize DOALL loops such as parallel for and, more recently, taskloop for task-based parallelism. On the other hand, when it is possible to prove that a loop is DOACROSS, programmers can try to parallelize it through parallel for and to use the OpenMP ordered directive to mark the region of the loop that has to be executed sequentially. However, when neither of the previous two cases can be proven, programmers have to be conservative and assume that the loop is DOACROSS (actually may DOACROSS). Previous work proposed speculative support for ta…
Advising OpenMP Parallelization via a Graph-Based Approach with Transformers
Ben-Gurion University of the Negev（内盖夫本-古里安大学）；Israel Atomic Energy Commission（以色列原子能委员会）；Intel Labs（英特尔实验室）；Nuclear Research Center – Negev（内盖夫核研究中心）；Technion – Israel Institute of Technology（以色列理工学院）
We present our experience with the modernization on the GR-MHD code BHAC, aimed at improving its novel hybrid (MPI+OpenMP) parallelization scheme. In doing so, we showcase the use of performance profiling tools usable on x86 (Intel-based) architectures. Our performance characterization and threading analysis provided guidance in improving the concurrency and thus the efficiency of the OpenMP parallel regions. We assess scaling and communication patterns in order to identify and alleviate MPI bottlenecks, with both runtime switches and precise code interventions. The performance of optimized ve…
Ben-Gurion University of the Negev（内盖夫本-古里安大学）；Nuclear Research Center – Negev（内盖夫核研究中心）；Technion – Israel Institute of Technology（以色列理工学院）
In past years, the world has switched to many-core and multi-core shared memory architectures. As a result, there is a growing need to utilize these architectures by introducing shared memory parallelization schemes to software applications. OpenMP is the most comprehensive API that implements such schemes, characterized by a readable interface. Nevertheless, introducing OpenMP into code is challenging due to pervasive pitfalls in management of parallel shared memory. To facilitate the performance of this task, many source-to-source (S2S) compilers have been created over the years, tasked with…
Currently, multi/many-core CPUs are considered standard in most types of computers including, mobile phones, PCs or supercomputers. However, the parallelization of applications as well as refactoring/design of applications for efficient hardware usage remains restricted to experts who have advanced technical knowledge and who can invest time tuning their software. In this context, the compilation community has proposed different methods for automatic parallelization, but their focus is traditionally on loops and nested loops with the support of polyhedral techniques. In this study, we propose …
In this paper, a work-optimal parallelization of Kostelec and Rockmore's well-known fast Fourier transform and its inverse on the three-dimensional rotation group SO(3) is designed, implemented, and tested. To this end, the sequential algorithms are reviewed briefly first. In the subsequent design and implementation of the parallel algorithms, we use the well-known Forster (PCAM) method and the OpenMP standard. The parallelization itself is based on symmetries of the underlying basis functions and a geometric approach in which the resulting index range is transformed in such a way that distinc…
We investigate a parallelization strategy for dense matrix factorization (DMF) algorithms, using OpenMP, that departs from the legacy (or conventional) solution, which simply extracts concurrency from a multithreaded version of BLAS. This approach is also different from the more sophisticated runtime-assisted implementations, which decompose the operation into tasks and identify dependencies via directives and runtime support. Instead, our strategy attains high performance by explicitly embedding a static look-ahead technique into the DMF code, in order to overcome the performance bottleneck o…
In this article, Conway's Game of Life using OpenMP parallel processing to simulate several different parallel methods, experimental performance results and compare to find the optimal solution of the parallelization of the Game of Life. Finally pointed out the importance of the design of parallel algorithms in solving the parallel problem.
Clusters of SMP nodes provide support for a wide diversity of parallel programming paradigms. Combining both shared memory and message passing parallelizations within the same application, the hybrid MPI-OpenMP paradigm is an emerging trend for parallel programming to fully exploit distributed shared-memory architecture. In this paper, we improve the performance of MPEG-2 encoder and n-body simulation by employing the hybrid MPI-OpenMP programming paradigm on SMP clusters. The hierarchical image data structure of the MPEG bit-stream is eminently suitable for the hybrid model to achieve multipl…
Approximation via sampling is a widespread technique whenever exact solutions are too expensive. In this paper, we present techniques for an efficient parallelization of adaptive (a. k. a. progressive) sampling algorithms on multi-threaded shared-memory machines. Our basic algorithmic technique requires no synchronization except for atomic load-acquire and store-release operations. It does, however, require O(n) memory per thread, where n is the size of the sampling state. We present variants of the algorithm that either reduce this memory consumption to O(1) or ensure that deterministic resul…
This article describes a method for creating applications for cluster computing systems using the parallel BSF skeleton based on the original BSF (Bulk Synchronous Farm) model of parallel computations developed by the author earlier. This model uses the master/slave paradigm. The main advantage of the BSF model is that it allows to estimate the scalability of a parallel algorithm before its implementation. Another important feature of the BSF model is the representation of problem data in the form of lists that greatly simplifies the logic of building applications. The BSF skeleton is designed…
The main computing tasks of a finite element code(FE) for solving partial differential equations (PDE's) are the algebraic system assembly and the iterative solver. This work focuses on the first task, in the context of a hybrid MPI+X paradigm. Although we will describe algorithms in the FE context, a similar strategy can be straightforwardly applied to other discretization methods, like the finite volume method. The matrix assembly consists of a loop over the elements of the MPI partition to compute element matrices and right-hand sides and their assemblies in the local system to each MPI par…
We propose a new hybrid topology optimization algorithm based on multigrid approach that combines the parallelization strategy of CPU using OpenMP and heavily multithreading capabilities of modern Graphics Processing Units (GPU). In addition to that significant computational efficiency in memory requirement has been achieved using homogenization strategy. The algorithm has been integrated with versitile computing platform of MATLAB for ease of use and customization. The bottlenecking repetitive solution of the state equation has been solved using an optimized geometric multigrid approach along…
Understanding and optimizing the properties of solar cells is becoming a key issue in the search for alternatives to nuclear and fossil energy sources. A theoretical analysis via numerical simulations involves solving Maxwell's Equations in discretized form and typically requires substantial computing effort. We start from a hybrid-parallel (MPI+OpenMP) production code that implements the Time Harmonic Inverse Iteration Method (THIIM) with Finite-Difference Frequency Domain (FDFD) discretization. Although this algorithm has the characteristics of a strongly bandwidth-bound stencil update schem…
Pacific Northwest National Laboratory（太平洋西北国家实验室）；University of Texas at Austin（德克萨斯大学奥斯汀分校）
We present a generic framework for parallel coordinate descent (CD) algorithms that includes, as special cases, the original sequential algorithms Cyclic CD and Stochastic CD, as well as the recent parallel Shotgun algorithm. We introduce two novel parallel algorithms that are also special cases---Thread-Greedy CD and Coloring-Based CD---and give performance measurements for an OpenMP implementation of these.
Splotch is a rendering algorithm for exploration and visual discovery in particle-based datasets coming from astronomical observations or numerical simulations. The strengths of the approach are production of high quality imagery and support for very large-scale datasets through an effective mix of the OpenMP and MPI parallel programming paradigms. This article reports our experiences in re-designing Splotch for exploiting emerging HPC architectures nowadays increasingly populated with GPUs. A performance model is introduced for data transfers, computations and memory access, to guide our re-f…
