The author's research of topologies of parallel computing systems and the tasks solved with them, including the corresponding tools of their modeling, is summarized in the present paper. The original topological model of such systems is presented based on the modified Amdahl law. It allowed formalizing the dependence of the necessary number of processors and the maximal distance between information-adjacent vertices in a graph on the directive values of acceleration or efficiency. The dependences of these values on the system interconnection topology and on the information graph of the parallel task are also formalized. The tools for a comparative evaluation of these dependences, topological criteria and the functions of scaling and fault-tolerant operation of parallel systems are based on the author|s technique of projective description of graphs and the algorithms used in it.
In competitive parallel computing, the identical copies of a code in a phase of a sequential program are assigned to processor cores and the result of the fastest core is adopted. In the literature, it is reported that a superlinear speedup can be achieved if there is an enough fluctuation among the execution times consumed by the cores. Competitive parallel computing is a promising approach to use a huge amount of cores effectively. However, there is few theoretical studies on speedups which can be achieved by competitive parallel computing at present. In this paper, we present a behavioral m…
Serverless computing is increasingly being used for parallel computing, which have traditionally been implemented as stateful applications. Executing complex, burst-parallel, directed acyclic graph (DAG) jobs poses a major challenge for serverless execution frameworks, which will need to rapidly scale and schedule tasks at high throughput, while minimizing data movement across tasks. We demonstrate that, for serverless parallel computations, decentralized scheduling enables scheduling to be distributed across Lambda executors that can schedule tasks in parallel, and brings multiple benefits, i…
Optimization of hybrid parallel application execution in heterogeneous high performance computing systems considering execution time and power consumption
Quantum computing represents a paradigm shift for computation requiring an entirely new computer architecture. However, there is much that can be learned from traditional classical computer engineering. In this paper, we describe the Parallel Research Kernels (PRK), a tool that was very useful for designing classical parallel computing systems. The PRK are simple kernels written to expose bottlenecks that limit classical parallel computing performance. We hypothesize that an analogous tool for quantum computing, Quantum Research Kernels (QRK), may similarly aid the co-design of software and ha…
Mass spectrometry, commonly used for protein identification, generates a massive number of spectra that need to be matched against a large database. In reality, most of them remain unidentified or mismatched due to unexpected post-translational modifications. Open modification search (OMS) has been proposed as a strategy to improve the identification rate by considering every possible change in spectra, but it expands the search space exponentially. In this work, we propose HyperOMS, which redesigns OMS based on hyperdimensional computing to cope with such challenges. Unlike existing algorithm…
This paper presents our solution for optimization task of the 3rd ACM-China IPCC. By the complexity analysis, we identified three time-consuming subroutines of original algorithm: marking edges, computing pseudo inverse and sorting edges. These subroutines becomes the main performance bottleneck owing to their super-linear time complexity. To address this, we proposed LGRASS, a linear graph spectral sparsification algorithm to run in strictly linear time. LGRASS takes advantage of spanning tree properties and efficient algorithms to optimize bottleneck subroutines. Furthermore, we crafted a pa…
Parallel Collaborative ADMM Privacy Computing and Adaptive GPU Acceleration for Distributed Edge Networks
College of Cyber Security, Jinan University（暨南大学网络空间安全学院）；Department of Electrical and Computer Engineering, Western University（韦仕敦大学电气与计算机工程系）；School of Computer Science, Guangzhou University（广州大学计算机科学与工程学院）
Aspects of compatibility of topologies of parallel computing systems and tasks are investigated. The introduction of appropriate indexes based on the original topological model of parallel computations and on the nontraditional description of a graph by its projections is proposed and elucidated. On the example of hypercubic computing system (CS) and tasks with ring and star information topologies, we demonstrate determining the indexes and their use in a comparative analysis of the applicability of interconnect with a given topology for solving tasks with the same and different types of infor…
This paper examines a new parallel computation model called bulk synchronous farm (BSF) that focuses on estimating the scalability of compute-intensive iterative algorithms aimed at cluster computing systems. In the BSF model, a computer is a set of processor nodes connected by a network and organized according to the master/slave paradigm. A cost metric of the BSF model is presented. This cost metric requires the algorithm to be represented in the form of operations on lists. This allows us to derive an equation that predicts the scalability boundary of a parallel program: the maximum number …
This article describes a method for creating applications for cluster computing systems using the parallel BSF skeleton based on the original BSF (Bulk Synchronous Farm) model of parallel computations developed by the author earlier. This model uses the master/slave paradigm. The main advantage of the BSF model is that it allows to estimate the scalability of a parallel algorithm before its implementation. Another important feature of the BSF model is the representation of problem data in the form of lists that greatly simplifies the logic of building applications. The BSF skeleton is designed…
In practice, standard scheduling of parallel computing jobs almost always leaves significant portions of the available hardware unused, even with many jobs still waiting in the queue. The simple reason is that the resource requests of these waiting jobs are fixed and do not match the available, unused resources. However, with alternative but existing and well-established techniques it is possible to achieve a fully automated, adaptive parallelism that does not need pre-set, fixed resources. Here, we demonstrate that such an adaptively parallel program can indeed fill in all such scheduling gap…
In the field of High Performance Computing, communications among processes represent a typical bottleneck for massively parallel scientific applications. Object of this research is the development of a network interface card with specific offloading capabilities that could help large scale simulations in terms of communication latency and scalability with the number of computing elements. In particular this work deals with the development of a double precision floating point complex arithmetic unit with a parallel-pipelined architecture, in order to implement a massively parallel computing sys…
RAFI -- A Ray/Work Forwarding Infrastructure for Data Parallel Multi-Node/Multi-GPU Computing
NVIDIA（英伟达）；Bilkent University（比尔肯特大学）；SCI Institute, University of Utah（犹他大学计算研究所）；University of Cologne（科隆大学）；NASA Ames Research Center（NASA艾姆斯研究中心）；IT4Innovations, VSB -Technical University of Ostrava（Ostrava理工大学IT4创新中心）
