It is very common to use dynamic methods to detect deadlocks in MPI programs for the reason that static methods have some restrictions. To guarantee high reliability of some important MPI-based application software, a model of MPI synchronization communication is abstracted and a type of static method is devised to examine deadlocks in such modes. The model has three forms with different complexity: sequential model, single-loop model and nested-loop model. Sequential model is a base for all models. Single-loop model must be treated with a special type of equation group and nested-loop model extends the methods for the other two models. A standard Java-based software framework originated from these methods is constructed for determining whether MPI programs are free from synchronization communication deadlocks. Our practice shows the software framework is better than those tools using dynamic methods because it can dig out all synchronization communication deadlocks before an MPI-based program goes into running.
A model of MPI synchronization communication programs is presented and its three basic simplified models are also defined. A series of theorems and methods for deciding whether deadlocks will occur among the three models are given and proved strictly. These theories and methods for simple models' deadlock detection are the necessary base for real MPI program deadlock detection. The methods are based on a static analysis through programs and with runtime detection in necessary cases and they are able to determine before compiling whether it will be deadlocked for two of the three basic models. …
Detecting deadlocks in MPI synchronization communication programs is very difficult and need building program models. All complex models are based on sequential models. The sequential model is mapped into a set of character strings and its deadlock detection problem is translated into an equivalent multi-queue string matching problem. An algorithm is devised and implemented to statically detect deadlocks in sequential models of MPI synchronization communication programs. The time and space complexity of the algorithm is O(n) where n is the amount of message in model. The algorithm is better th…
The Message Passing Interface specification (MPI) defines a portable message-passing API used to program parallel computers. MPI programs manifest a number of challenges on what concerns correctness: sent and expected values in communications may not match, resulting in incorrect computations possibly leading to crashes; and programs may deadlock resulting in wasted resources. Existing tools are not completely satisfactory: model-checking does not scale with the number of processes; testing techniques wastes resources and are highly dependent on the quality of the test set. As an alternative, …
Program correctness is one of the most difficult challenges in parallel programming. Message Passing Interface MPI is widely used in writing parallel applications. Since MPI is not a compiled language, the programmer will be enfaced with several programming bugs.This paper presents the most common programming bugs arise in MPI programs to help the programmer to compromise between the advantage of parallelism and the extra effort needed to detect and fix such bugs. An algebraic specification of an MPI-like programming language, called Simple MPI (SMPI), to be used in writing MPI programs specif…
Message Passing Interfaces (MPI) plays an important role in parallel computing. Many parallel applications are implemented as MPI programs. The existing methods of bug detection for MPI programs have the shortage of providing both input and non-determinism coverage, leading to missed bugs. In this paper, we employ symbolic execution to ensure the input coverage, and propose an on-the-fly schedule algorithm to reduce the interleaving explorations for non-determinism coverage, while ensuring the soundness and completeness. We have implemented our approach as a tool, called MPISE, which can autom…
The Message Passing Interface (MPI) is the de facto standard message-passing infrastructure for developing parallel applications. Two decades after the first version of the library specification, MPI-based applications are nowadays routinely deployed on super and cluster computers. These applications, written in C or Fortran, exhibit intricate message passing behaviours, making it hard to statically verify important properties such as the absence of deadlocks. Our work builds on session types, a theory for describing protocols that provides for correct-by-construction guarantees in this regard…
Task graphs have been studied for decades as a foundation for scheduling irregular parallel applications and incorporated in programming models such as OpenMP. While many high-performance parallel libraries are based on task graphs, they also have additional scheduling requirements, such as synchronization from inner levels of data parallelism and internal blocking communications. In this paper, we extend task-graph scheduling to support efficient synchronization and communication within tasks. Our scheduler avoids deadlock and oversubscription of worker threads, and refines victim selection t…
Power and energy consumption is becoming key challenges to deploy the first exascale supercomputer successfully. Large-scale HPC applications waste a significant amount of power in communication and synchronization-related idle times. However, due to the time scale at which communication happens, transitioning in low power states during communication's idle times may introduce unacceptable overhead in applications' execution time. In this paper, we present COUNTDOWN, a runtime library, supported by a methodology and analysis tool for identifying and automatically reducing the power consumption…
Co-Design and Evaluation of a CPU-Free MPI GPU Communication Abstraction and Implementation
University of New Mexico（新墨西哥大学）；Oak Ridge National Laboratory（橡树岭国家实验室）；Tennessee Tech University（田纳西科技大学）；University of Alabama（阿拉巴马大学）；Center for Computing Research Sandia National Laboratories（桑迪亚国家实验室计算研究中心）
Distributed Order Recording Techniques for Efficient Record-and-Replay of Multi-threaded Programs
Nanchang Hangkong University（南昌航空大学）；Pacific Northwest National Laboratory（太平洋西北国家实验室）；RIKEN R-CCS（理化学研究所计算科学研究中心）；NVIDIA（英伟达）；Lawrence Livermore National Laboratory（劳伦斯利弗莫尔国家实验室）；Technical University of Munich（慕尼黑工业大学）
Multi-Variant Execution Environments (MVEEs) are a promising technique to protect software against memory corruption attacks. They transparently execute multiple, diversified variants (often referred to as replicae) of the software receiving the same inputs. By enforcing and monitoring the lock-step execution of the replicae's system calls, and by deploying diversity techniques that prevent an attacker from simultaneously compromising multiple replicae, MVEEs can block attacks before they succeed. Existing MVEEs cannot handle non-trivial multi-threaded programs because their undeterministic be…
Polyhedral compilers perform optimizations such as tiling and parallelization; when doing both, they usually generate code that executes "barrier-synchronized wavefronts" of tiles. We present a system to express and generate code for hybrid schedules, where some constraints are automatically satisfied through the structure of the code, and the remainder are dynamically enforced at run-time with data flow mechanisms. We prove bounds on the added overheads that are better, by at least one polynomial degree, than those of previous techniques. We propose a generic mechanism to implement the needed…
The performance of highly parallel applications on distributed-memory systems is influenced by many factors. Analytic performance modeling techniques aim to provide insight into performance limitations and are often the starting point of optimization efforts. However, coupling analytic models across the system hierarchy (socket, node, network) fails to encompass the intricate interplay between the program code and the hardware, especially when execution and communication bottlenecks are involved. In this paper we investigate the effect of "bottleneck evasion" and how it can lead to automatic o…
