Recently, the research on local repair codes is mainly confined to repair the failed nodes within each repair group. But if the extreme cases occur that the entire repair group has failed, the local code stored in the failed group need to be recovered as a whole. In this paper, local codes with cooperative repair, in which the local codes are constructed based on minimum storage regeneration (MSR) codes, is proposed to achieve repairing the failed groups. Specifically, the proposed local codes with cooperative repair construct a kind of mutual interleaving structure among the parity symbols, that the parity symbols of each local code, named as distributed local parity, can be generated by the parity symbols of the MSR codes in its two adjacent local codes. Taking advantage of the structure given, the failed local groups can be repaired cooperatively by their adjacent local groups with lower repair locality, and meanwhile the minimum distance of local codes with cooperative repair is derived. Theoretical analysis and simulation experiments show that, compared with codes with local regeneration (such as MSR-local codes and MBR-local codes), the proposed local codes with cooperative repair have benefits in bandwidth overhead and repair locality for the case of local groups failure.
Cooperative regenerating codes are designed for repairing multiple node failures in distributed storage systems. In contrast to the original repair model of regenerating codes, which are for the repair of single node failure, data exchange among the new nodes is enabled. It is known that further reduction in repair bandwidth is possible with cooperative repair. Currently in the literature, we have an explicit construction of exact-repair cooperative code achieving all parameters corresponding to the minimum-bandwidth point. We give a slightly generalized and more flexible version of this coope…
The optimal tradeoff between node storage and repair bandwidth is an important issue for distributed storage systems (DSSs). As for realistic DSSs with clusters, when repairing a failed node, it is more efficient to download more data from intra-cluster nodes than from cross-cluster nodes. Therefore, it is meaningful to differentiate the repair bandwidth from intra-cluster and cross-cluster. For cluster DSSs the tradeoff has been considered with special repair assumptions where all the alive nodes are utilized to repair a failed node. In this paper, we investigate the optimal tradeoff for clus…
Erasure-correcting codes, that support local repair of codeword symbols, have attracted substantial attention recently for their application in distributed storage systems. This paper investigates a generalization of the usual locally repairable codes. In particular, this paper studies a class of codes with the following property: any small set of codeword symbols can be reconstructed (repaired) from a small number of other symbols. This is referred to as cooperative local repair. The main contribution of this paper is bounds on the trade-off of the minimum distance and the dimension of such c…
Dau et al. recently extend Guruswami and Wootters' scheme (STOC'2016) to cooperatively repair two or three erasures in Reed-Solomon (RS) codes. However, their scheme restricts to either the case that the characteristic of $F$ divides the extension degree $[F\!:\!B]$ or some special failure patterns, where $F$ is the base field of the RS code and $B$ is the subfield of the repair symbols. In this paper, we derive an improved cooperative repair scheme that removes all these restrictions. That is, our scheme applies to any characteristic of $F$ and can repair all failure patterns of two or three …
Repair of multiple partially failed cache nodes is studied in a distributed wireless content caching system, where $r$ out of a total of $n$ cache nodes lose part of their cached data. Broadcast repair of failed cache contents at the network edge is studied; that is, the surviving cache nodes transmit broadcast messages to the failed ones, which are then used, together with the surviving data in their local cache memories, to recover the lost content. The trade-off between the storage capacity and the repair bandwidth is derived. It is shown that utilizing the broadcast nature of the wireless …
In this paper, a new repair scheme for a modified construction of MDS codes is studied. The obtained repair scheme has optimal bandwidth for multiple failed nodes under the cooperative repair model. In addition, the repair scheme has relatively low access property, where the number of data accessed is less than two times the optimal value.
Two widely studied models of multiple-node repair in distributed storage systems are centralized repair and cooperative repair. The centralized model assumes that all the failed nodes are recreated in one location, while the cooperative one stipulates that the failed nodes may communicate but are distinct, and the amount of data exchanged between them is included in the repair bandwidth. As our first result, we prove a lower bound on the minimum bandwidth of cooperative repair. We also show that the cooperative model is stronger than the centralized one, in the sense that any MDS code with opt…
In wireless distributed storage systems, storage nodes are connected by wireless channels, which are broadcast in nature. This paper exploits this unique feature to design an efficient repair mechanism, called broadcast repair, for wireless distributed storage systems in the presence of multiple-node failures. Due to the broadcast nature of wireless transmission, we advocate a new measure on repair performance called repair-transmission bandwidth. In contrast to repair bandwidth, which measures the average number of packets downloaded by a newcomer to replace a failed node, repair-transmission…
In wireless distributed storage systems, storage nodes are connected by wireless channels, which are broadcast in nature. This paper exploits this unique feature to design an efficient repair mechanism, called broadcast repair, for wireless distributed storage systems with multiple-node failures. Since wireless channels are typically bandwidth limited, we advocate a new measure on repair performance called repair-transmission bandwidth, which measures the average number of packets transmitted by helper nodes per failed node. The fundamental tradeoff between storage amount and repair-transmissi…
We consider a distributed storage system with $n$ nodes, where a user can recover the stored file from any $k$ nodes, and study the problem of repairing $r$ partially failed nodes. We consider \textit{broadcast repair}, that is, $d$ surviving nodes transmit broadcast messages on an error-free wireless channel to the $r$ nodes being repaired, which are then used, together with the surviving data in the local memories of the failed nodes, to recover the lost content. First, we derive the trade-off between the storage capacity and the repair bandwidth for partial repair of multiple failed nodes, …
Recently we constructed an explicit family of locally repairable and locally regenerating codes. Their existence was proven by Kamath et al. but no explicit construction was given. Our design is based on HashTag codes that can have different sub-packetization levels. In this work we emphasize the importance of having two ways to repair a node: repair only with local parity nodes or repair with both local and global parity nodes. We say that the repair strategy is network traffic driven since it is in connection with the concrete system and code parameters: the repair bandwidth of the code, the…
In this paper, we find the Laplace Stieltjes transform of the probability of data loss for the k-out-of-n distributed storage system with deterministic repair times. We consider two repair models, namely the serial and parallel repair. We show that for failure rate much lower than the repair rate, mean time of data loss for the two models is the same unlike the case for exponential repair models.
Cooperative repair model is an available technology to deal with multiple node failures in distributed storage systems. Recently, explicit constructions of cooperative MSR codes were given by Ye (IEEE Transactions on Information Theory, 2020) with sub-packetization level $(d-k+h)(d-k+1)^n$. Specifically, the sub-packetization level is $(h+1)2^n$ when $d=k+1$. In this paper, we propose a new cooperative repair scheme by means of the inter-instance and intra-instance pairing inherited from the perfect code which reduces the sub-packetization to $2^n$ when $(h+1)|2^n$ and $(2\ell+1)2^n$ when $h+1…
In distributed storage systems, cooperative regenerating codes tradeoff storage for repair bandwidth in the case of multiple node failures. In rack-aware distributed storage systems, there is no cost associated with transferring symbols within a rack. Hence, the repair bandwidth will only take into account cross-rack transfer. Rack-aware regenerating codes for the case of single node failures have been studied and their repair bandwidth tradeoff characterized. In this paper, we consider the framework of rack-aware cooperative regenerating codes for the case of multiple node failures where the …
In this paper, we consider heterogeneous distributed storage systems (DSSs) having flexible reconstruction degree, where each node in the system has dynamic repair bandwidth and dynamic storage capacity. In particular, a data collector can reconstruct the file at time $t$ using some arbitrary nodes in the system and for a node failure the system can be repaired by some set of arbitrary nodes. Using $min$-$cut$ bound, we investigate the fundamental tradeoff between storage and repair cost for our model of heterogeneous DSS. In particular, the problem is formulated as bi-objective optimization l…
We study the trade-off between storage overhead and inter-cluster repair bandwidth in clustered storage systems, while recovering from multiple node failures within a cluster. A cluster is a collection of $m$ nodes, and there are $n$ clusters. For data collection, we download the entire content from any $k$ clusters. For repair of $t \geq 2$ nodes within a cluster, we take help from $\ell$ local nodes, as well as $d$ helper clusters. We characterize the optimal trade-off under functional repair, and also under exact repair for the minimum storage and minimum inter-cluster bandwidth (MBR) opera…
Distributed storage systems suffer from significant repair traffic generated due to frequent storage node failures. This paper shows that properly designed low-density parity-check (LDPC) codes can substantially reduce the amount of required block downloads for repair thanks to the sparse nature of their factor graph representation. In particular, with a careful construction of the factor graph, both low repair-bandwidth and high reliability can be achieved for a given code rate. First, a formula for the average repair bandwidth of LDPC codes is developed. This formula is then used to establis…
One of the primary objectives of a distributed storage system is to reliably store a large amount $dsize$ of source data for a long duration using a large number $N$ of unreliable storage nodes, each with capacity $nsize$. The storage overhead $β$ is the fraction of system capacity available beyond $dsize$, i.e., $β= 1- \frac{dsize}{N \cdot nsize}$. Storage nodes fail randomly over time and are replaced with initially empty nodes, and thus data is erased from the system at an average rate $erate = λ\cdot N \cdot nsize$, where $1/λ$ is the average lifetime of a node before failure. To maintain …
In this paper, we consider the setting of exact repair linear regenerating codes. Under this setting, we derive a new outer bound on the storage-repair-bandwidth trade-off for the case when $d = k = n -1$, where $(n, k, d)$ are parameters of the regenerating code, with their usual meaning. Taken together with the achievability result of Tian et. al. [1], we show that the new outer bound derived here completely characterizes the trade-off for the case of exact repair linear regenerating codes, when $d = k = n -1$. The new outer bound is derived by analyzing the dual code of the linear regenerat…
Distributed storage systems and associated storage codes can efficiently store a large amount of data while ensuring that data is retrievable in case of node failure. The study of such systems, particularly the design of storage codes over finite fields, assumes that the physical channel through which the nodes communicate is error-free. This is not always the case, for example, in a wireless storage system. We study the probability that a subpacket is repaired incorrectly during node repair in a distributed storage system, in which the nodes communicate over an AWGN or Rayleigh fading channel…
A major issue of locally repairable codes is their robustness. If a local repair group is not able to perform the repair process, this will result in increasing the repair cost. Therefore, it is critical for a locally repairable code to have multiple repair groups. In this paper we consider robust locally repairable coding schemes which guarantee that there exist multiple distinct (not necessarily disjoint) alternative local repair groups for any single failure such that the failed node can still be repaired locally even if some of the repair groups are not available. We use linear programming…
A new system model reflecting the clustered structure of distributed storage is suggested to investigate interplay between storage overhead and repair bandwidth as storage node failures occur. Large data centers with multiple racks/disks or local networks of storage devices (e.g. sensor network) are good applications of the suggested clustered model. In realistic scenarios involving clustered storage structures, repairing storage nodes using intact nodes residing in other clusters is more bandwidth-consuming than restoring nodes based on information from intra-cluster nodes. Therefore, it is i…
A new system model reflecting the clustered structure of distributed storage is suggested to investigate bandwidth requirements for repairing failed storage nodes. Large data centers with multiple racks/disks or local networks of storage devices (e.g. sensor network) are good applications of the suggested clustered model. In realistic scenarios involving clustered storage structures, repairing storage nodes using intact nodes residing in other clusters is more bandwidth-consuming than restoring nodes based on information from intra-cluster nodes. Therefore, it is important to differentiate bet…
We consider the problem of multiple-node repair in distributed storage systems under the cooperative model, where the repair bandwidth includes the amount of data exchanged between any two different storage nodes. Recently, explicit constructions of MDS codes with optimal cooperative repair bandwidth for all possible parameters were given by Ye and Barg (IEEE Transactions on Information Theory, 2019). The node size (or sub-packetization) in this construction scales as $\exp(Θ(n^h))$, where $h$ is the number of failed nodes and $n$ is the code length. In this paper, we give new explicit constru…
We consider distributed storage (DS) for a wireless network where mobile devices arrive and depart according to a Poisson random process. Content is stored in a number of mobile devices, using an erasure correcting code. When requesting a piece of content, a user retrieves the content from the mobile devices using device-to-device communication or, if not possible, from the base station (BS), at the expense of a higher communication cost. We consider the repair problem when a device that stores data leaves the network. In particular, we introduce a repair scheduling where repair is performed (…
Distributed storage codes have important applications in the design of modern storage systems. In a distributed storage system, every storage node has a probability to fail and once an individual storage node fails, it must be reconstructed using data stored in the surviving nodes. Computation load and network bandwidth are two important issues we need to concern when repairing a failed node. The traditional maximal distance separable (MDS) storage codes have low repair complexity but high repair bandwidth. On the contrary, minimal storage regenerating (MSR) codes have low repair bandwidth but…
We study a generalization of the setting of regenerating codes, motivated by applications to storage systems consisting of clusters of storage nodes. There are $n$ clusters in total, with $m$ nodes per cluster. A data file is coded and stored across the $mn$ nodes, with each node storing $α$ symbols. For availability of data, we require that the file be retrievable by downloading the entire content from any subset of $k$ clusters. Nodes represent entities that can fail. We distinguish between intra-cluster and inter-cluster bandwidth (BW) costs during node repair. Node-repair in a cluster is a…
We introduce the Fixed Cluster Repair System (FCRS) as a novel architecture for Distributed Storage Systems (DSS), achieving a small repair bandwidth while guaranteeing a high availability. Specifically we partition the set of servers in a DSS into $s$ clusters and allow a failed server to choose any cluster other than its own as its repair group. Thereby, we guarantee an availability of $s-1$. We characterize the repair bandwidth vs. storage trade-off for the FCRS under functional repair and show that the minimum repair bandwidth can be improved by an asymptotic multiplicative factor of $2/3$…
In distributed storage systems (DSSs), the optimal tradeoff between node storage and repair bandwidth is an important issue for designing distributed coding strategies to ensure large scale data reliability. The capacity of DSSs is obtained as a function of node storage and repair bandwidth parameters, characterizing the tradeoff. There are lots of works on DSSs with clusters (racks) where the repair bandwidths from intra-cluster and cross-cluster are differentiated. However, separate nodes are also prevalent in the realistic DSSs, but the works on DSSs with clusters and separate nodes (CSN-DS…
High availability of containerized applications requires to perform robust storage of applications' state. Since basic replication techniques are extremely costly at scale, storage space requirements can be reduced by means of erasure or repairing codes. In this paper we address storage regeneration using repair codes, a robust distributed storage technique with no need to fully restore the whole state in case of failure. In fact, only the lost servers' content is replaced. To do so, new cleanslate storage units are made operational at a cost for activating new storage servers and a cost for t…
Here, we revisit the problem of exploring the secrecy capacity of minimum storage cooperative regenerating (MSCR) codes under the $\{l_1,l_2\}$-eavesdropper model, where the eavesdropper can observe the data stored on $l_1$ nodes and the repair downloads of an additional $l_2$ nodes. Compared to minimum storage regenerating (MSR) codes which support only single node repairs, MSCR codes allow efficient simultaneous repairs of multiple failed nodes, referred to as a \emph{repair group}. However, the repair data sent from a helper node to another failed node may vary with different repair groups …
