High-performance computing systems (HPC) provide powerful capabilities for modeling, simulation, and data analytics for a broad class of computational problems. They enable extreme performance of the order of quadrillion floating-point arithmetic calculations per second by aggregating the power of millions of compute, memory, networking and storage components. With the rapidly growing scale and complexity of HPC systems for achieving even greater performance, ensuring their reliable operation in the face of system degradations and failures is a critical challenge. System fault events often lead the scientific applications to produce incorrect results, or may even cause their untimely termination. The sheer number of components in modern extreme-scale HPC systems and the complex interactions and dependencies among the hardware and software components, the applications, and the physical environment makes the design of practical solutions that support fault resilience a complex undertaking. To manage this complexity, we developed a methodology for designing HPC resilience solutions using design patterns. We codified the well-known techniques for handling faults, errors and failures that have been devised, applied and improved upon over the past three decades in the form of design patterns. In this paper, we present a pattern language to enable a structured approach to the development of HPC resilience solutions. The pattern language reveals the relations among the resilience patterns and provides the means to explore alternative techniques for handling a specific fault model that may have different efficiency and complexity characteristics. Using the pattern language enables the design and implementation of comprehensive resilience solutions as a set of interconnected resilience patterns that can be instantiated across layers of the system stack.
Flexible Coded Distributed Convolution Computing for Enhanced Straggler Resilience and Numerical Stability in Distributed CNNs
College of Computer Science, Chongqing University(重庆大学计算机学院);School of Electrical Engineering, Korea University(韩国大学电气工程学院);School of Electronic Information and Communications, Huazhong University of Science and Technology(华中科技大学电子信息与通信学院);City University of Hong Kong(香港城市大学)
Resilient by Design -- Active Inference for Distributed Continuum Intelligence
Department of Computer Systems and Sciences, Stockholm University(斯德哥尔摩大学计算机系统与科学系);Department of Information Engineering, University of Pisa(比萨大学信息工程系);Distributed Systems Group, TU Wien(维也纳技术大学分布式系统组)