Synchronizing clocks in distributed systems is well-understood, both in terms of fault-tolerance in fully connected systems and the dependence of local and global worst-case skews (i.e., maximum clock difference between neighbors and arbitrary pairs of nodes, respectively) on the diameter of fault-free systems. However, so far nothing non-trivial is known about the local skew that can be achieved in topologies that are not fully connected even under a single Byzantine fault. Put simply, in this work we show that the most powerful known techniques for fault-tolerant and gradient clock synchronization are compatible, in the sense that the best of both worlds can be achieved simultaneously.
Concretely, we combine the Lynch-Welch algorithm [Welch1988] for synchronizing a clique of $n$ nodes despite up to $f<n/3$ Byzantine faults with the gradient clock synchronization (GCS) algorithm by Lenzen et al. [Lenzen2010] in order to render the latter resilient to faults. As this is not possible on general graphs, we augment an input graph $\mathcal{G}$ by replacing each node by $3f+1$ fully connected copies, which execute an instance of the Lynch-Welch algorithm. We then interpret these clusters as supernodes executing the GCS algorithm, where for each cluster its correct nodes' Lynch-Welch clocks provide estimates of the logical clock of the supernode in the GCS algorithm. By connecting clusters corresponding to neighbors in $\mathcal{G}$ in a fully bipartite manner, supernodes can inform each other about (estimates of) their logical clock values. This way, we achieve asymptotically optimal local skew, granted that no cluster contains more than $f$ faulty nodes, at factor $O(f)$ and $O(f^2)$ overheads in terms of nodes and edges, respectively. Note that tolerating $f$ faulty neighbors trivially requires degree larger than $f$, so this is asymptotically optimal as well.
In this paper we suggest a method by which reference broadcast synchronization (RBS), and other methods of estimating clock values, can be incorporated in standard clock synchronization algorithms to improve synchronization quality. We advocate a logical separation of the task of estimating the clock values of other nodes in the network from the task of using these estimates to output a logical clock value. The separation is achieved by means of a virtual estimate graph, overlaid on top of the real network graph, which represents the information various nodes can obtain about each other. RBS e…
We study the problem of clock synchronization in highly dynamic networks, where communication links can appear or disappear at any time. The nodes in the network are equipped with hardware clocks, but the rate of the hardware clocks can vary arbitrarily within specific bounds, and the estimates that nodes can obtain about the clock values of other nodes are inherently inaccurate. Our goal in this setting is to output a logical clock at each node such that the logical clocks of any two nodes are not too far apart, and nodes that remain close to each other in the network for a long time are bett…
Sub-nanosecond precision clock synchronization over the packet network has been achieved by the White Rabbit protocol for a decade. However, few computer systems utilize such a technique. We try to attract more interest in the clock synchronization problem. We first introduce the basics of clock and synchronization in the time and frequency discipline. Then we revisit several related works, such as Google's Spanner, Huygens, FARMv2, DTP, and Sundial, explain why these works could be improved. Finally, we briefly discuss an independent time network approach towards low-cost and high-precision s…
Virtualization of distributed real-time systems enables the consolidation of mixed-criticality functions on a shared hardware platform thus easing system integration. Time-triggered communication and computation can act as an enabler of safe hard real-time systems. A time-triggered hypervisor that activates virtual CPUs according to a global schedule can provide the means to allow for a resource efficient implementation of the time-triggered paradigm in virtualized distributed real-time systems. A prerequisite of time-triggered virtualization for hard real-time systems is providing access to a…
Design of clock synchronization for networked nodes faces a fundamental trade-off between synchronization accuracy and universality for heterogeneous platforms, because a high synchronization accuracy generally requires platform-dependent hardware-level network packet timestamping. This paper presents TouchSync, a new indoor clock synchronization approach for wearables that achieves millisecond accuracy while preserving universality in that it uses standard system calls only, such as reading system clock, sampling sensors, and sending/receiving network messages. The design of TouchSync is driv…
For tolerating Byzantine faults of both the terminal and communication components in self-stabilizing clock synchronization, the two-dimensional self-stabilizing Byzantine-fault-tolerant clock synchronization problem is investigated and solved. By utilizing the time-triggered (TT) stage provided in the underlying networks as TT communication windows, the approximate agreement, hopping procedure, and randomized grandmasters are integrated into the overall solution. It is shown that with partitioning the communication components into $3$ arbitrarily connected subnetworks, efficient synchronizati…
We revisit the approach to Byzantine fault-tolerant clock synchronization based on approximate agreement introduced by Lynch and Welch. Our contribution is threefold: (1) We provide a slightly refined variant of the algorithm yielding improved bounds on the skew that can be achieved and the sustainable frequency offsets. (2) We show how to extend the technique to also synchronize clock rates. This permits less frequent communication without significant loss of precision, provided that clock rates change sufficiently slowly. (3) We present a coupling scheme that allows to make these algorithms …
Robust pulse synchronization is fundamental in constructing reliable synchronous applications in wired and wireless distributed systems. In wired systems, self-stabilizing Byzantine pulse synchronization aims for synchronizing fault-prone distributed components with arbitrary initial states in bounded-delay message-passing systems. In wireless systems, fault-tolerant synchronization of pulse-coupled oscillators is also built for a similar goal but often works under specific system restrictions, such as low computation power, low message complexity, and anonymous physical pulses whose senders c…
We present concept and implementation of a self-stabilizing Byzantine fault-tolerant distributed clock generation scheme for multi-synchronous GALS architectures in critical applications. It combines a variant of a recently introduced self-stabilizing algorithm for generating low-frequency, low-accuracy synchronized pulses with a simple non-stabilizing high-frequency, high-accuracy clock synchronization algorithm. We provide thorough correctness proofs and a performance analysis, which use methods from fault-tolerant distributed computing research but also addresses hardware-related issues lik…
In this article, we present a new control theoretic distributed time synchronization algorithm, named PISync, in order to synchronize sensor nodes in Wireless Sensor Networks (WSNs). PISync algorithm is based on a Proportional-Integral (PI) controller. It applies a proportional feedback (P) and an integral feedback (I) on the local measured synchronization errors to compensate the differences between the clock offsets and clock speeds. We present practical flooding-based and fully distributed protocol implementations of the PISync algorithm, and we provide theoretical analysis to highlight the…
Recent studies exploited external periodic synchronous signals to synchronize a pair of network nodes to address a threat of delaying the communications between the nodes. However, the sensing-based synchronization may yield faults due to nonmalicious signal and sensor noises. This paper considers a system of N nodes that will fuse their peer-to-peer synchronization results to correct the faults. Our analysis gives the lower bound of the number of faults that the system can tolerate when N is up to 12. If the number of faults is no greater than the lower bound, the faults can be identified and…
We introduce a graph-theoretic approach to synchronizing clocks in an {\em ad hoc} network of $N$~timepieces. Clocks naturally drift away from being synchronized because of many physical factors. The manual way of clock synchronization suffers from an inherrent propagation of the so called "clock drift" due to "word-of-mouth effect." The current standard way of automated clock synchronization is either via radio band transmission of the global clock or via the software-based Network Time Protocol (NTP). Synchronization via radio band transmission suffers from the wave transmission delay, while…
For reaching dependable high-precision clock synchronization (CS) upon IoT networks, the distributed CS paradigm adopted in ultra-high reliable systems and the master-slave CS paradigm adopted in high-performance but unreliable systems are integrated. Meanwhile, traditional internal clock synchronization is also integrated with external time references to achieve efficient stabilization. Low network connectivity, low complexity, high precision, and high reliability are all considered. To tolerate permanent failures, the Byzantine CS is integrated with the common CS protocols. To tolerate trans…
We present a detailled timed automata model of the clock synchronization algorithm that is currently being used in a wireless sensor network (WSN) that has been developed by the Dutch company Chess. Using the Uppaal model checker, we establish that in certain cases a static, fully synchronized network may eventually become unsynchronized if the current algorithm is used, even in a setting with infinitesimal clock drifts.
Accurate and energy-efficient clock synchronization is an enabler for many applications of Wireless Sensor Networks. A fine-grained synchronization is beneficial both at the system level, for example to favor deterministic radio protocols, and at the application level, when network-wide event timestamping is required. However, there is a tradeoff between the resolution of a WSN node's timekeeping device and its energy consumption. The Virtual High-resolution Timer (VHT) is an innovative solution, that was proposed to overcome this tradeoff. It combines a high-resolution oscillator to a low-pow…
Synchronizing clocks in a distributed system in which processes communicate through messages with uncertain delays is subject to inherent errors. Prior work has shown upper and lower bounds on the best synchronization achievable in a variety of network topologies and assumptions about the uncertainty on the message delays. However, until now there has not been a tight closed-form expression for the optimal synchronization in $k$-ary $m$-cubes with wraparound, where $k$ is odd. In this paper, we prove a lower bound of $\frac{1}{4}um\left(k-\frac{1}{k}\right)$, where $k$ is the (odd) number of p…
We consider the clock synchronization problem in the (discrete) beeping model: Given a network of $n$ nodes with each node having a clock value $δ(v) \in \{0,\ldots T-1\}$, the goal is to synchronize the clock values of all nodes such that they have the same value in any round. As is standard in clock synchronization, we assume \emph{arbitrary activations} for all nodes, i.e., the nodes start their protocol at an arbitrary round (not limited to $\{0,\ldots,T-1\}$). We give an asymptotically optimal algorithm that runs in $4D + \Bigl\lfloor \frac{D}{\lfloor T/4 \rfloor} \Bigr \rfloor \cdot (T \…
We consider the External Clock Synchronization problem in dynamic sensor networks. Initially, sensors obtain inaccurate estimations of an external time reference and subsequently collaborate in order to synchronize their internal clocks with the external time. For simplicity, we adopt the drift-free assumption, where internal clocks are assumed to tick at the same pace. Hence, the problem is reduced to an estimation problem, in which the sensors need to estimate the initial external time. This work is further relevant to the problem of collective approximation of environmental values by biolog…
Consider an asynchronous network in a shared-memory environment consisting of n nodes. Assume that up to f of the nodes might be Byzantine (n > 12f), where the adversary is full-information and dynamic (sometimes called adaptive). In addition, the non-Byzantine nodes may undergo transient failures. Nodes advance in atomic steps, which consist of reading all registers, performing some calculation and writing to all registers. This paper contains three main contributions. First, the clock-function problem is defined, which is a generalization of the clock synchronization problem. This generaliza…
We consider the fundamental problem of clock synchronization in a synchronous multi-agent system. Each agent holds a clock with an arbitrary initial value, and clocks must eventually indicate the same value. Previous algorithms worked in static networks with drastic connectivity properties and assumed that global information is available at each agent. In this paper, we propose different solutions for time-varying topologies that require neither strong connectivity nor any global knowledge on the network. First, we study the case of unbounded clocks, and propose a self-stabilizing $MinMax$ alg…
The vast majority of hardware architectures use a carefully timed reference signal to clock their computational logic. However, standard distribution solutions are not fault-tolerant. In this work, we present a simple grid structure as a more reliable clock propagation method and study it by means of simulation experiments. Fault-tolerance is achieved by forwarding clock pulses on arrival of the second of three incoming signals from the previous layer. A key question is how well neighboring grid nodes are synchronized, even without faults. Analyzing the clock skew under typical-case conditions…
Distributed system applications rely on a fine-grain common sense of time. Existing systems maintain the common sense of time by keeping each independent machine as close as possible to wall-clock time through a combination of software protocols like NTP and GPS signals and/or precision references like atomic clocks. This approach is expensive and has tolerance limitations that require protocols to deal with asynchrony and its performance consequences. Moreover, at data-center scale it is impractical to distribute a physical clock as is done on a chip or printed circuit board. In this paper we…
Cryptographic signatures can be used to increase the resilience of distributed systems against adversarial attacks, by increasing the number of faulty parties that can be tolerated. While this is well-studied for consensus, it has been underexplored in the context of fault-tolerant clock synchronization, even in fully connected systems. Here, the honest parties of an $n$-node system are required to compute output clocks of small skew (i.e., maximum phase offset) despite local clock rates varying between $1$ and $\vartheta>1$, end-to-end communication delays varying between $d-u$ and $d$, and t…
