首页/论文/1908.01455

arXiv 1908.014552019-08-05cs.DC

The fault-tolerant cluster-sending problem

Jelle Hellings, Mohammad Sadoghi

备注：: A brief announcement of this work will be presented at the 33rd International Symposium on Distributed Computing (DISC 2019)

英文摘要：

The development of fault-tolerant distributed systems that can tolerate Byzantine behavior has traditionally been focused on consensus protocols, which support fully-replicated designs. For the development of more sophisticated high-performance Byzantine distributed systems, more specialized fault-tolerant communication primitives are necessary, however. In this paper, we identify an essential communication primitive and study it in depth. In specifics, we formalize the cluster-sending problem, the problem of sending a message from one Byzantine cluster to another Byzantine cluster in a reliable manner. We not only formalize this fundamental problem, but also establish lower bounds on the complexity of this problem under crash failures and Byzantine failures. Furthermore, we develop practical cluster-sending protocols that meet these lower bounds and, hence, have optimal complexity. As such, our work provides a strong foundation for the further exploration of novel designs that address challenges encountered in fault-tolerant distributed systems.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

阅读原文

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

本文内容

英文摘要相关论文

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

继续探索

相关论文

Fault Tolerant Leader Election in Distributed Systems

There are many distributed systems which use a leader in their logic. When such systems need to be fault tolerant and the current leader suffers a technical problem, it is necesary to apply a special algorithm in order to choose a new leader. In this paper I present a new fault tolerant algorithm which elects a new leader based on a random roulette wheel selection.

2017-03-07cs.DC

Checkpoint-Restart Libraries Must Become More Fault Tolerant

Production MPI codes need checkpoint-restart (CPR) support. Clearly, checkpoint-restart libraries must be fault tolerant lest they open up a window of vulnerability for failures with byzantine outcomes. But, certain popular libraries that leverage MPI are evidently not fault tolerant. Nowadays, fault detection with automatic recovery without batch requeueing is a strong requirement for production environments. Thus, allowing deadlock and setting long timeouts are suboptimal for fault detection even when paired with conservative recovery from the penultimate checkpoint. When MPI is used as a co…

2021-12-20cs.DC

Recent Results on Fault-Tolerant Consensus in Message-Passing Networks

Fault-tolerant consensus has been studied extensively in the literature, because it is one of the most important distributed primitives and has wide applications in practice. This paper surveys important results on fault-tolerant consensus in message-passing networks, and the focus is on results from the past decade. Particularly, we categorize the results into two groups: new problem formulations and practical applications. In the first part, we discuss new ways to define the consensus problem, which includes larger input domains, link fault models, different network models . . . etc, and bri…

2016-08-29cs.DC

What does fault tolerant Deep Learning need from MPI?

Deep Learning (DL) algorithms have become the de facto Machine Learning (ML) algorithm for large scale data analysis. DL algorithms are computationally expensive - even distributed DL implementations which use MPI require days of training (model learning) time on commonly studied datasets. Long running DL applications become susceptible to faults - requiring development of a fault tolerant system infrastructure, in addition to fault tolerant DL algorithms. This raises an important question: What is needed from MPI for de- signing fault tolerant DL implementations? In this paper, we address thi…

2017-09-11cs.DC

Hosting Byzantine Fault Tolerant Services on a Chord Ring

In this paper we demonstrate how stateful Byzantine Fault Tolerant services may be hosted on a Chord ring. The strategy presented is fourfold: firstly a replication scheme that dissociates the maintenance of replicated service state from ring recovery is developed. Secondly, clients of the ring based services are made replication aware. Thirdly, a consensus protocol is introduced that supports the serialization of updates. Finally Byzantine fault tolerant replication protocols are developed that ensure the integrity of service data hosted on the ring.

2010-06-17cs.DC

A Fault Tolerant Mechanism for Partitioning and Offloading Framework in Pervasive Environments

Application partitioning and code offloading are being researched extensively during the past few years. Several frameworks for code offloading have been proposed. However, fewer works attempted to address issues occurred with its implementation in pervasive environments such as frequent network disconnection due to high mobility of users. Thus, in this paper, we proposed a fault tolerant algorithm that helps in consolidating the efficiency and robustness of application partitioning and offloading frameworks. To permit the usage of different fault tolerant policies such as replication and chec…

2021-09-19cs.DC

更多相关论文收起更多论文

Proactive Service Migration for Long-Running Byzantine Fault Tolerant Systems

In this paper, we describe a novel proactive recovery scheme based on service migration for long-running Byzantine fault tolerant systems. Proactive recovery is an essential method for ensuring long term reliability of fault tolerant systems that are under continuous threats from malicious adversaries. The primary benefit of our proactive recovery scheme is a reduced vulnerability window. This is achieved by removing the time-consuming reboot step from the critical path of proactive recovery. Our migration-based proactive recovery is coordinated among the replicas, therefore, it can automatica…

2008-03-11cs.DC

On the Byzantine-Fault-Tolerant Consensus for Blockchain among Connected Vehicles

It is a critical matter for a blockchain system whether a Byzantine fault tolerance (BFT) can be guaranteed during a consensus process. Can connected vehicles (CVs) achieve the BFT consensus when the vehicles keep mobile? This paper seeks an answer to this fundamental question. It focuses on characterizing the impact of mobility on the performance of a BFT consensus among CVs.

2021-12-25cs.DC

A Hypermedia Distributed Application for Monitoring and Fault-Injection in Embedded Fault-tolerant Parallel Programs

We describe a distributed, multimedia application which is being developed in the framework of the ESPRIT-IV Project 21012 EFTOS (Embedded Fault-Tolerant Supercomputing). The application dynamically sets up a hierarchy of HTML pages reflecting the current status of an EFTOS-compliant dependable application running on a Parsytec CC system. These pages are fed to a World-Wide Web browser playing the role of a hypermedia monitor. The adopted approach allows the user to concentrate on the high-level aspects of his/her application so to quickly assess the quality of its current fault-tolerance desi…

2014-01-13cs.DC

Fault Tolerant QR Factorization for General Matrices

This paper presents a fault-tolerant algorithm for the QR factorization of general matrices. It relies on the communication-avoiding algorithm, and uses the structure of the reduction of each part of the computation to introduce redundancies that are sufficient to recover the state of a failed process. After a process has failed, its state can be recovered based on the data held by one process only. Besides, it does not add any significant operation in the critical path during failure-free execution.

2016-04-14cs.DC

Revisiting EZBFT: A Decentralized Byzantine Fault Tolerant Protocol with Speculation

In this note, we revisit EZBFT[2] and present safety, liveness and execution consistency violations in the protocol. To demonstrate these violations, we present simple scenarios, involving only four replicas, two clients, and one or two owner changes. We also note shortcomings of the presented TLA+ specification used to model check the proposed protocol.

2019-09-09cs.DC

Holistic Approach for Fault-Tolerant Network-on-Chip based Many-Core Systems

In this paper we describe a holistic approach for Fault-Tolerant Network-on-Chip (NoC) based many-core systems that incorporates a System Health Monitoring Unit (SHMU) which collects all the fault information from the system, classifies them and provides different solutions for different fault classes. A Mapper/Scheduler Unit (MSU) is used for online generation of different mapping and scheduling solutions based on the current fault configuration of the system. For detection of faults, we have leveraged concurrent online checkers, able to capture faults with low detection latency and providing…

2016-01-26cs.DC

Fault Tolerant Consensus Agreement Algorithm

Recently a new fault tolerant and simple mechanism was designed for solving commit consensus problem. It is based on replicated validation of messages sent between transaction participants and a special dispatcher validator manager node. This paper presents a correctness, safety proofs and performance analysis of this algorithm.

2017-06-11cs.DBcs.DC

Byzantine Cluster-Sending in Expected Constant Communication

Traditional resilient systems operate on fully-replicated fault-tolerant clusters, which limits their scalability and performance. One way to make the step towards resilient high-performance systems that can deal with huge workloads, is by enabling independent fault-tolerant clusters to efficiently communicate and cooperate with each other, as this also enables the usage of high-performance techniques such as sharding and parallel processing. Recently, such inter-cluster communication was formalized as the Byzantine cluster-sending problem, and worst-case optimal protocols have been proposed t…

2021-08-19cs.DBcs.DC

A Game-theoretic Approach for Synthesizing Fault-Tolerant Embedded Systems

In this paper, we present an approach for fault-tolerant synthesis by combining predefined patterns for fault-tolerance with algorithmic game solving. A non-fault-tolerant system, together with the relevant fault hypothesis and fault-tolerant mechanism templates in a pool are translated into a distributed game, and we perform an incomplete search of strategies to cope with undecidability. The result of the game is translated back to executable code concretizing fault-tolerant mechanisms using constraint solving. The overall approach is implemented to a prototype tool chain and is illustrated u…

2010-11-01cs.DCcs.GTcs.SE

Can Agent Intelligence be used to Achieve Fault Tolerant Parallel Computing Systems?

The work reported in this paper is motivated towards validating an alternative approach for fault tolerance over traditional methods like checkpointing that constrain efficacious fault tolerance. Can agent intelligence be used to achieve fault tolerant parallel computing systems? If so, "What agent capabilities are required for fault tolerance?", "What parallel computational tasks can benefit from such agent capabilities?" and "How can agent capabilities be implemented for fault tolerance?" need to be addressed. Cognitive capabilities essential for achieving fault tolerance through agents are …

2013-08-13cs.DCcs.MA

A Byzantine Fault Tolerant Distributed Commit Protocol

In this paper, we present a Byzantine fault tolerant distributed commit protocol for transactions running over untrusted networks. The traditional two-phase commit protocol is enhanced by replicating the coordinator and by running a Byzantine agreement algorithm among the coordinator replicas. Our protocol can tolerate Byzantine faults at the coordinator replicas and a subset of malicious faults at the participants. A decision certificate, which includes a set of registration records and a set of votes from participants, is used to facilitate the coordinator replicas to reach a Byzantine agree…

2007-08-01cs.DBcs.DC

Starting a Dialog between Model Checking and Fault-tolerant Distributed Algorithms

Fault-tolerant distributed algorithms are central for building reliable spatially distributed systems. Unfortunately, the lack of a canonical precise framework for fault-tolerant algorithms is an obstacle for both verification and deployment. In this paper, we introduce a new domain-specific framework to capture the behavior of fault-tolerant distributed algorithms in an adequate and precise way. At the center of our framework is a parameterized system model where control flow automata are used for process specification. To account for the specific features and properties of fault-tolerant dis…

2012-10-14cs.DCcs.FL

拜占庭容错分布式系统中获得信念的充分条件

A Sufficient Condition for Gaining Belief in Byzantine Fault-Tolerant Distributed Systems

本文针对拜占庭容错分布式系统，提出智能体从自身历史推断正确信念的充分条件及有限时间检查过程，为高效协议开发与正确性证明奠定基础。

2023-07-11cs.DCcs.MA

SeeMoRe: A Fault-Tolerant Protocol for Hybrid Cloud Environments

Large scale data management systems utilize State Machine Replication to provide fault tolerance and to enhance performance. Fault-tolerant protocols are extensively used in the distributed database infrastructure of large enterprises such as Google, Amazon, and Facebook, as well as permissioned blockchain systems like IBM's Hyperledger Fabric. However, and in spite of years of intensive research, existing fault-tolerant protocols do not adequately address all the characteristics of distributed system applications. In particular, hybrid cloud environments consisting of private and public cloud…

2019-06-18cs.DBcs.DC

Efficient and Simple Algorithms for Fault Tolerant Spanners

It was recently shown that a version of the greedy algorithm gives a construction of fault-tolerant spanners that is size-optimal, at least for vertex faults. However, the algorithm to construct this spanner is not polynomial-time, and the best-known polynomial time algorithm is significantly suboptimal. Designing a polynomial-time algorithm to construct (near-)optimal fault-tolerant spanners was given as an explicit open problem in the two most recent papers on fault-tolerant spanners ([Bodwin, Dinitz, Parter, Vassilevka Williams SODA '18] and [Bodwin, Patel PODC '19]). We give a surprisingly…

2020-05-24cs.DCcs.DMcs.DS