In a realistic distributed storage environment, storage nodes are usually placed in racks, a metallic support designed to accommodate electronic equipment. It is known that the communication (bandwidth) cost between nodes which are in the same rack is much lower than between nodes which are in different racks.
In this paper, a new model, where the storage nodes are placed in two racks, is proposed and analyzed. Moreover, the two-rack model is generalized to any number of racks. In this model, the storage nodes have different repair costs depending on the rack where they are placed. A threshold function, which minimizes the amount of stored data per node and the bandwidth needed to regenerate a failed node, is shown. This threshold function generalizes the ones given for previous distributed storage models. The tradeoff curve obtained from this threshold function is compared with the ones obtained from the previous models, and it is shown that this new model outperforms the previous ones in terms of repair cost.
The majority of works in distributed storage networks assume a simple network model with a collection of identical storage nodes with the same communication cost between the nodes. In this paper, we consider a realistic multi-rack distributed data storage network and present a code design framework for this model. Considering the cheaper data transmission within the racks, our code construction method is able to locally repair the nodes failure within the same rack by using only the survived nodes in the same rack. However, in the case of severe failure patterns when the information content of…
The paper is devoted to the problem of erasure coding in distributed storage. We consider a model of storage that assumes that nodes are organized into equally sized groups, called racks, that within each group the nodes can communicate freely without taxing the system bandwidth, and that the only information transmission that counts is the one between the racks. This assumption implies that the nodes within each of the racks can collaborate before providing information to the failed node. The main emphasis of the paper is on code construction for this storage model. We present an explicit fam…
In distributed storage systems, cooperative regenerating codes tradeoff storage for repair bandwidth in the case of multiple node failures. In rack-aware distributed storage systems, there is no cost associated with transferring symbols within a rack. Hence, the repair bandwidth will only take into account cross-rack transfer. Rack-aware regenerating codes for the case of single node failures have been studied and their repair bandwidth tradeoff characterized. In this paper, we consider the framework of rack-aware cooperative regenerating codes for the case of multiple node failures where the …
Erasure coding is widely used for massive storage in data centers to achieve high fault tolerance and low storage redundancy. Since the cross-rack communication cost is often high, it is critical to design erasure codes that minimize the cross-rack repair bandwidth during failure repair. In this paper, we analyze the optimal trade-off between storage redundancy and cross-rack repair bandwidth specifically for data centers, subject to the condition that the original data can be reconstructed from a sufficient number of any non-failed nodes. We characterize the optimal trade-off curve under func…
In a modern distributed storage system, storage nodes are organized in racks, and the cross-rack communication dominates the system bandwidth. In We study the rack-aware storage system where all storage nodes are organized in racks and within each rack the nodes can communicate freely without taxing the system bandwidth. Rack-aware regenerating codes (RRCs) were proposed for minimizing the repair bandwidth for single erasures. In the initial setting of RRCs, the repair of a single node requires the participation of all the remaining nodes in the rack containing the failed node as well as a lar…
In large data centers, storage nodes are organized in racks, and the cross-rack transmission dominates the bandwidth cost. For the repair of single node failures, codes achieving the tradeoff between the storage redundancy and cross-rack repair bandwidth are called rack-aware regenerating codes (RRCs). In this work, we give the first explicit construction of RRCs with the minimum repair bandwidth (i.e., the cross-rack bandwidth equals the storage size per node). Our construction applies to all admissible parameters and has the lowest sub-packetization level. Moreover, the underlying finite fie…
We consider the rack-aware storage system where \(n\) nodes are organized in \(\bar{n}\) racks each containing \(u\) nodes, and any \(k\) nodes can retrieve the stored file. Moreover, any single node erasure can be recovered by downloading data from \(\bar{d}\) helper racks as well as the remaining \(u\!-\!1\) nodes in the same rack. Previous work mostly focuses on minimizing the cross-rack repair bandwidth under the condition \(\bar{d}\geq \bar{k}\), where \(\bar{k}=\lfloor\frac{k}{u}\rfloor\). However, \(\bar{d}\geq \bar{k}\) is not an intrinsic condition for the rack-aware storage model. In…
We consider the rack-aware storage system where $n=\bar{n}u$ nodes are organized in $\bar{n}$ racks each containing $u$ nodes, and any $k=\bar{k}u+u_0~(0\leq u_0<u)$ nodes can retrieve the original data file. More importantly, the cross-rack communication cost is much more expensive than the intra-rack communication cost, so that the latter is usually neglected in the system bandwidth. The MSRR (minimum storage rack-aware regenerating) code is an important variation of regenerating codes that achieves the optimal repair bandwidth for single node failures in the rack-aware model. However, expli…
The minimum storage rack-aware regenerating (MSRR) code is a variation of regenerating codes that achieves the optimal repair bandwidth for a single node failure in the rack-aware model. The authors in~\cite{Chen-Barg2019} and~\cite{Zhou-Zhang2021} provided explicit constructions of MSRR codes for all parameters to repair a single failed node. This paper generalizes the results in~\cite{Chen-Barg2019} to the case of multiple node failures. We propose a class of MDS array codes and scalar Reed-Solomon (RS) codes, and show that these codes have optimal repair bandwidth and error resilient capabi…
In modern practical data centers, storage nodes are usually organized into equally sized groups, which is called racks. The cost of cross-rack communication is much more expensive compared with the intra-rack communication cost. The codes for this system are called rack-aware regenerating codes. Similar to standard minimum storage regenerating (MSR) codes, it is a challenging task to construct minimum storage rack-aware regenerating (MSRR) codes achieving the cut-set bound. The known constructions of MSRR codes achieving the cut-set bound give codes with alphabet size $q$ exponential in the co…
In distributed storage systems (DSSs), the optimal tradeoff between node storage and repair bandwidth is an important issue for designing distributed coding strategies to ensure large scale data reliability. The capacity of DSSs is obtained as a function of node storage and repair bandwidth parameters, characterizing the tradeoff. There are lots of works on DSSs with clusters (racks) where the repair bandwidths from intra-cluster and cross-cluster are differentiated. However, separate nodes are also prevalent in the realistic DSSs, but the works on DSSs with clusters and separate nodes (CSN-DS…
A new system model reflecting the clustered structure of distributed storage is suggested to investigate interplay between storage overhead and repair bandwidth as storage node failures occur. Large data centers with multiple racks/disks or local networks of storage devices (e.g. sensor network) are good applications of the suggested clustered model. In realistic scenarios involving clustered storage structures, repairing storage nodes using intact nodes residing in other clusters is more bandwidth-consuming than restoring nodes based on information from intra-cluster nodes. Therefore, it is i…
The optimal tradeoff between node storage and repair bandwidth is an important issue for distributed storage systems (DSSs). As for realistic DSSs with clusters, when repairing a failed node, it is more efficient to download more data from intra-cluster nodes than from cross-cluster nodes. Therefore, it is meaningful to differentiate the repair bandwidth from intra-cluster and cross-cluster. For cluster DSSs the tradeoff has been considered with special repair assumptions where all the alive nodes are utilized to repair a failed node. In this paper, we investigate the optimal tradeoff for clus…
