This paper studies the problem of code symbol availability: a code symbol is said to have $(r, t)$-availability if it can be reconstructed from $t$ disjoint groups of other symbols, each of size at most $r$. For example, $3$-replication supports $(1, 2)$-availability as each symbol can be read from its $t= 2$ other (disjoint) replicas, i.e., $r=1$. However, the rate of replication must vanish like $\frac{1}{t+1}$ as the availability increases.
This paper shows that it is possible to construct codes that can support a scaling number of parallel reads while keeping the rate to be an arbitrarily high constant. It further shows that this is possible with the minimum distance arbitrarily close to the Singleton bound. This paper also presents a bound demonstrating a trade-off between minimum distance, availability and locality. Our codes match the aforementioned bound and their construction relies on combinatorial objects called resolvable designs.
From a practical standpoint, our codes seem useful for distributed storage applications involving hot data, i.e., the information which is frequently accessed by multiple processes in parallel.
In this work codes with availability are constructed based on the cyclic \emph{locally repairable code} (LRC) construction by Tamo et al. and their extension to $(r,ρ)$-locality by Chen et al. The minimum distance of these codes is increased by carefully extending their defining set. We give a bound on the dimension of LRCs with availability and orthogonal repair sets and show that the given construction is optimal for a range of parameters.
Ever-increasing amounts of data are created and processed in internet-scale companies such as Google, Facebook, and Amazon. The efficient storage of such copious amounts of data has thus become a fundamental and acute problem in modern computing. No single machine can possibly satisfy such immense storage demands. Therefore, distributed storage systems (DSS), which rely on tens of thousands of storage nodes, are the only viable solution. Such systems are broadly used in all modern internet-scale systems. However, the design of a DSS poses a number of crucial challenges, markedly different from…
Maximally recoverable codes with locality and availability
IMUVa-Mathematics Research Institute, University of Valladolid（瓦拉多利德大学数学研究所）；Signal Processing and Communications Research Center (SPCRC), International Institute of Information Technology, Hyderabad（海得拉巴国际信息学院信号处理与通信研究中心）
In this work we present a class of locally recoverable codes, i.e. codes where an erasure at a position $P$ of a codeword may be recovered from the knowledge of the entries in the positions of a recovery set $R_P$. The codes in the class that we define have availability, meaning that for each position $P$ there are several distinct recovery sets. Also, the entry at position $P$ may be recovered even in the presence of erasures in some of the positions of the recovery sets, and the number of supported erasures may vary among the various recovery sets.
We introduce the Fixed Cluster Repair System (FCRS) as a novel architecture for Distributed Storage Systems (DSS), achieving a small repair bandwidth while guaranteeing a high availability. Specifically we partition the set of servers in a DSS into $s$ clusters and allow a failed server to choose any cluster other than its own as its repair group. Thereby, we guarantee an availability of $s-1$. We characterize the repair bandwidth vs. storage trade-off for the FCRS under functional repair and show that the minimum repair bandwidth can be improved by an asymptotic multiplicative factor of $2/3$…
The paper presents techniques for analyzing the expected download time in distributed storage systems that employ systematic availability codes. These codes provide access to hot data through the systematic server containing the object and multiple recovery groups. When a request for an object is received, it can be replicated (forked) to the systematic server and all recovery groups. We first consider the low-traffic regime and present the close-form expression for the download time. By comparison across systems with availability, maximum distance separable (MDS), and replication codes, we de…
In this paper, we study vector codes with all-symbol locality, where the local code is either a Minimum Bandwidth Regenerating (MBR) code or a Minimum Storage Regenerating (MSR) code. In the first part, we present vector codes with all-symbol MBR locality, for all parameters, that have both optimal minimum-distance and optimal rate. These codes combine ideas from two popular codes in the distributed storage literature, Product-Matrix codes and Tamo-Barg codes. In the second part which deals with codes having all-symbol MSR locality, we follow a Pairwise Coupling Transform-based approach to arr…
We extend the notion of locality from the Hamming metric to the rank and subspace metrics. Our main contribution is to construct a class of array codes with locality constraints in the rank metric. Our motivation for constructing such codes stems from designing codes for efficient data recovery from correlated and/or mixed (i.e., complete and partial) failures in distributed storage systems. Specifically, the proposed local rank-metric codes can recover locally from 'crisscross errors and erasures', which affect a limited number of rows and/or columns of the storage system. We also derive a Si…
Recently, Skabelund defined new maximal curves which are cyclic extensions of the Suzuki and Ree curves. Previously, the now well-known GK curves were found as cyclic extensions of the Hermitian curve. In this paper, we consider locally recoverable codes constructed from these new curves, complementing that done for the GK curve. Locally recoverable codes allow for the recovery of a single symbol by accessing only a few others which form what is known as a recovery set. If every symbol has at least two disjoint recovery sets, the code is said to have availability. Three constructions are descr…
In order to accommodate the ever-growing data from various, possibly independent, sources and the dynamic nature of data usage rates in practical applications, modern cloud data storage systems are required to be scalable, flexible, and heterogeneous. Codes with hierarchical locality have been intensively studied due to their effectiveness in reducing the average reading time in cloud storage. In this paper, we present the first codes with hierarchical locality that achieve scalability and flexibility in heterogeneous cloud storage using small field size. We propose a double-level construction…
The cover metric is suitable for describing the resilience against correlated errors in arrays, in particular crisscross errors, which makes it interesting for applications such as distributed data storage (DDS). In this work, we consider codes designed for the cover metric that have locality, that means lost symbols can be recovered by using only a few other (local) symbols. We derive and prove a Singleton-like bound on the minimum cover distance of cover-metric codes with locality and propose a bound-achieving construction. Further, we explore the performance of our construction in compariso…
Locally recoverable (LRC) codes and their variants have been extensively studied in recent years. In this paper we focus on cyclic constructions of LRC codes and derive conditions on the zeros of the code that support the property of hierarchical locality. As a result, we obtain a general family of hierarchical LRC codes for a new range of code parameters. We also observe that our approach enables one to represent an LRC code in quasicyclic form, and use this representation to construct tail-biting convolutional LRC codes with locality. Among other results, we extend the general approach to cy…
When a node in a distributed storage system fails, it needs to be promptly repaired to maintain system integrity. While typical erasure codes can provide a significant storage advantage over replication, they suffer from poor repair efficiency. Locally repairable codes (LRCs) tackle this issue by reducing the number of nodes participating in the repair process (locality), at the cost of reduced minimum distance. In this paper, we study the tradeoff between locality and minimum distance of LRCs with local codes that have arbitrary distance requirements. Unlike existing methods where both the lo…
In cloud storage systems, hot data is usually replicated over multiple nodes in order to accommodate simultaneous access by multiple users as well as increase the fault tolerance of the system. Recent cloud storage research has proposed using availability codes, which is a special class of erasure codes, as a more storage efficient way to store hot data. These codes enable data recovery from multiple, small disjoint groups of servers. The number of the recovery groups is referred to as the availability and the size of each group as the locality of the code. Until now, we have very limited know…
Locally recoverable (LRC) codes provide ways of recovering erased coordinates of the codeword without having to access each of the remaining coordinates. A subfamily of LRC codes with hierarchical locality (H-LRC codes) provides added flexibility to the construction by introducing several tiers of recoverability for correcting different numbers of erasures. We present a general construction of codes with 2-level hierarchical locality from maps between algebraic curves and specialize it to several code families obtained from quotients of curves by a subgroup of the automorphism group, including…
Locally recoverable codes (LRCs) were proposed for the recovery of data in distributed and cloud storage systems about nine years ago. A lot of progress on the study of LRCs has been made by now. However, there is a lack of general theory on the minimum linear locality of linear codes. In addition, the minimum linear locality of many known families of linear codes is not studied in the literature. Motivated by these two facts, this paper develops some general theory about the minimum linear locality of linear codes, and investigates the minimum linear locality of a number of families of linear…
Locally repairable codes(LRCs) play important roles in distributed storage systems(DSS). LRCs with small locality have their own advantages since fewer available symbols are needed in the recovery of erased symbols. In this paper, we prove an upper bound on the dimension of LRCs with minimum distance $d\geq 7$. An upper bound on the length of almost optimal LRCs with $d=7$, $r=2$ at $q^2+q+3$ is proved. Then based on the $t$-spread structure, we give an algorithm to construct almost optimal LRCs with $d=7$, $r=2$ and length $n\geq 3\lceil\frac{\sqrt{2}q}{3}\rceil$ when $q\geq 4$, whose dimensi…
A new type of spatially coupled low-density parity-check (SC-LDPC) codes motivated by practical storage applications is presented. SC-LDPCL codes (suffix 'L' stands for locality) can be decoded locally at the level of sub-blocks that are much smaller than the full code block, thus offering flexible access to the coded information alongside the strong reliability of the global full-block decoding. Toward that, we propose constructions of SC-LDPCL codes that allow controlling the trade-off between local and global correction performance. In addition to local and global decoding, the paper develo…
An index code is said to be locally decodable if each receiver can decode its demand using its side information and by querying only a subset of the transmitted codeword symbols instead of observing the entire codeword. Local decodability can be a beneficial feature in some communication scenarios, such as when the receivers can afford to listen to only a part of the transmissions because of limited availability of power. The locality of an index code is the ratio of the maximum number of codeword symbols queried by a receiver to the message length. In this paper we analyze the optimum localit…
A code is called a locally repairable code (LRC) if any code symbol is a function of a small fraction of other code symbols. When a locally repairable code is employed in a distributed storage systems, an erased symbol can be recovered by accessing only a small number of other symbols, and hence alleviating the network resources required during the repair process. In this paper we consider repeated-root constacyclic codes, which is a generalization of cyclic codes, that are optimal with respect to a Singleton-like bound on minimum distance. An LRC with the structure of a constacyclic code can …
In distributed storage systems, both the repair bandwidth and locality are important repair cost metrics to evaluate the performance of a storage code. Recently, Guruswami and Wooters proposed an optimal linear repair scheme based on Reed-Solomon codes for a single failure, improved the bandwidth of the classical repair scheme. In this paper, we consider the repair bandwidth of Generalized Reed-Muller (GRM) codes, which have good locality property. We generalize Guruswami and Wooters' repairing scheme to GRM codes for single failure, which has nontrivial bandwidth closing to the lower bound wh…
Data centres that use consumer-grade disks drives and distributed peer-to-peer systems are unreliable environments to archive data without enough redundancy. Most redundancy schemes are not completely effective for providing high availability, durability and integrity in the long-term. We propose alpha entanglement codes, a mechanism that creates a virtual layer of highly interconnected storage devices to propagate redundant information across a large scale storage system. Our motivation is to design flexible and practical erasure codes with high fault-tolerance to improve data durability and …
