Partial MDS (PMDS) and sector-disk (SD) codes are classes of erasure correcting codes that combine locality with strong erasure correction capabilities. We construct PMDS and SD codes with local regeneration where each local code is a bandwidth-optimal regenerating MDS code. In the event of a node failure, these codes reduce both, the number of servers that have to be contacted as well as the amount of network traffic required for the repair process. The constructions require significantly smaller field size than the only other construction known in literature. Further, we present a construction of PMDS codes with global regeneration which allow to efficiently repair patterns of node failures that exceed the local erasure correction capability of the code and thereby invoke repair across different local groups.
Partial MDS (PMDS) and sector-disk (SD) codes are classes of erasure codes that combine locality with strong erasure correction capabilities. We construct PMDS and SD codes where each local code is a bandwidth-optimal regenerating MDS code. The constructions require significantly smaller field size than the only other construction known in literature.
Distributed storage plays a crucial role in the current cloud computing framework. After the theoretical bound for distributed storage was derived by the pioneer work of the regenerating code, Reed-Solomon code based regenerating codes were developed. The RS code based minimum storage regeneration code (RS-MSR) and the minimum bandwidth regeneration code (RS-MBR) can achieve theoretical bounds on the MSR point and the MBR point respectively in code regeneration. They can also maintain the MDS property in code reconstruction. However, in the hostile network where the storage nodes can be compro…
The high repair cost of (n,k) Maximum Distance Separable (MDS) erasure codes has recently motivated a new class of codes, called Regenerating Codes, that optimally trade off storage cost for repair bandwidth. On one end of this spectrum of Regenerating Codes are Minimum Storage Regenerating (MSR) codes that can match the minimum storage cost of MDS codes while also significantly reducing repair bandwidth. In this paper, we describe Exact-MSR codes which allow for any failed nodes (whether they are systematic or parity nodes) to be regenerated exactly rather than only functionally or informatio…
We consider error decoding of locally repairable codes (LRC) and partial MDS (PMDS) codes through interleaved decoding. For a specific class of LRCs we investigate the success probability of interleaved decoding. For PMDS codes we show that there is a wide range of parameters for which interleaved decoding can increase their decoding radius beyond the minimum distance with the probability of successful decoding approaching $1$, when the code length goes to infinity.
A family of codes with a natural two-dimensional structure is presented, inspired by an application of RAID type of architectures whose units are solid state drives (SSDs). Arrays of SSDs behave differently to arrays of hard disk drives (HDDs), since hard errors in sectors are common and traditional RAID approaches (like RAID 5 or RAID 6) may be either insufficient or excessive. An efficient solution to this problem is given by the new codes presented, called partial-MDS (PMDS) codes.
This work deals with partial MDS (PMDS) codes, a special class of locally repairable codes, used for distributed storage system. We first show that a known construction of these codes, using Gabidulin codes, can be extended to use any maximum rank distance code. Then we define a standard form for the generator matrices of PMDS codes and use this form to give an algebraic description of PMDS generator matrices. This implies that over a sufficiently large finite field a randomly chosen generator matrix in PMDS standard form generates a PMDS code with high probability. This also provides sufficie…
Partial MDS (PMDS) codes are erasure codes combining local (row) correction with global additional correction of entries, while Sector-Disk (SD) codes are erasure codes that address the mixed failure mode of current RAID systems. It has been an open problem to construct general codes that have the PMDS and the SD properties, and previous work has relied on Monte-Carlo searches. In this paper, we present a general construction that addresses the case of any number of failed disks and in addition, two erased sectors. The construction requires a modest field size. This result generalizes previous…
We consider a set up where a file of size M is stored in n distributed storage nodes, using an (n,k) minimum storage regenerating (MSR) code, i.e., a maximum distance separable (MDS) code that also allows efficient exact-repair of any failed node. The problem of interest in this paper is to minimize the repair bandwidth B for exact regeneration of a single failed node, i.e., the minimum data to be downloaded by a new node to replace the failed node by its exact replica. Previous work has shown that a bandwidth of B=[M(n-1)]/[k(n-k)] is necessary and sufficient for functional (not exact) regene…
Partial-MDS (PMDS) codes are a family of locally repairable codes, mainly used for distributed storage. They are defined to be able to correct any pattern of $s$ additional erasures, after a given number of erasures per locality group have occurred. This makes them also maximally recoverable (MR) codes, another class of locally repairable codes. It is known that MR codes in general, and PMDS codes in particular, exist for any set of parameters, if the field size is large enough. Moreover, some explicit constructions of PMDS codes are known, mostly (but not always) with a strong restriction on …
We propose reducible algebraic curves as a mechanism to construct Partial MDS (PMDS) codes geometrically. We obtain new general existence results, new explicit constructions and improved estimates on the smallest field sizes over which such codes can exist. Our results are obtained by combining ideas from projective algebraic geometry, combinatorics and probability theory.
In this work it is shown that locally repairable codes (LRCs) can be list-decoded efficiently beyond the Johnson radius for a large range of parameters by utilizing the local error-correction capabilities. The corresponding decoding radius is derived and the asymptotic behavior is analyzed. A general list-decoding algorithm for LRCs that achieves this radius is proposed along with an explicit realization for LRCs that are subcodes of Reed--Solomon codes (such as, e.g., Tamo--Barg LRCs). Further, a probabilistic algorithm of low complexity for unique decoding of LRCs is given and its success pr…
We study $q$-ary codes with distance defined by a partial order of the coordinates of the codewords. Maximum Distance Separable (MDS) codes in the poset metric have been studied in a number of earlier works. We consider codes that are close to MDS codes by the value of their minimum distance. For such codes, we determine their weight distribution, and in the particular case of the "ordered metric" characterize distributions of points in the unit cube defined by the codes. We also give some constructions of codes in the ordered Hamming space.
In this paper, we establish the Singleton bound for pomset block codes ($(Pm,π)$-codes) of length $N$ over the ring $\mathbb{Z}_m$. We give a necessary condition for a code to be MDS in the pomset (block) metric and prove that every MDS $(Pm,π)$-code is an MDS $(P,π)$-code. Then we proceed on to find $I$-perfect and $r$-perfect codes. Further, given an ideal with partial and full counts, we look into how MDS and $I$-perfect codes relate to one another. For chain pomset, we obtain the duality theorem for pomset block codes of length $N$ over $\mathbb{Z}_m$; and, the weight distribution of MDS p…
This paper presents an explicit construction for an $((n,k,d=n-1), (α,β))$ regenerating code over a field $\mathbb{F}_Q$ operating at the Minimum Storage Regeneration (MSR) point. The MSR code can be constructed to have rate $k/n$ as close to $1$ as desired, sub-packetization given by $r^{\frac{n}{r}}$, for $r=(n-k)$, field size no larger than $n$ and where all code symbols can be repaired with the same minimum data download. The construction modifies a prior construction by Sasidharan et. al. which required far larger field-size. A building block appearing in the construction is a scalar MDS …
A code construction and repair scheme for optimal functional regeneration of multiple node failures is presented, which is based on stitching together short MDS codes on carefully chosen sets of points lying on a linearized polynomial. The nodes are connected wirelessly, hence all transmissions by helper nodes during a repair round are available to all the nodes being repaired. The scheme is simple and practical because of low subpacketization, low I/O cost and low computational cost. Achievability of the minimum-bandwidth regenerating (MBR) point, as well as an interior point, on the optimal …
Distributed matrix computations over large clusters can suffer from the problem of slow or failed worker nodes (called stragglers) which can dominate the overall job execution time. Coded computation utilizes concepts from erasure coding to mitigate the effect of stragglers by running 'coded' copies of tasks comprising a job; stragglers are typically treated as erasures. While this is useful, there are issues with applying, e.g., MDS codes in a straightforward manner. Several practical matrix computation scenarios involve sparse matrices. MDS codes typically require dense linear combinations o…
In order to scale economically, data centers are increasingly evolving their data storage methods from the use of simple data replication to the use of more powerful erasure codes, which provide the same level of reliability as replication but at a significantly lower storage cost. In particular, it is well known that Maximum-Distance-Separable (MDS) codes, such as Reed-Solomon codes, provide the maximum storage efficiency. While the use of codes for providing improved reliability in archival storage systems, where the data is less frequently accessed (or so-called "cold data"), is well unders…
The MDS property (aka the $k$-out-of-$n$ property) requires that if a file is split into several symbols and subsequently encoded into $n$ coded symbols, each being stored in one storage node of a distributed storage system (DSS), then an user can recover the file by accessing any $k$ nodes. We study the so-called $p$-decodable $μ$-secure erasure coding scheme $(1 \leq p \leq k - μ, 0 \leq μ< k, p | (k-μ))$, which satisfies the MDS property and the following additional properties: (P1) strongly secure up to a threshold: an adversary which eavesdrops at most $μ$ storage nodes gains no informati…
For a linear block code C, its stopping redundancy is defined as the smallest number of check nodes in a Tanner graph for C, such that there exist no stopping sets of size smaller than the minimum distance of C. Schwartz and Vardy conjectured that the stopping redundancy of an MDS code should only depend on its length and minimum distance. We define the (n,t)-single-exclusion number, S(n,t) as the smallest number of t-subsets of an n-set, such that for each i-subset of the n-set, i=1,...,t+1, there exists a t-subset that contains all but one element of the i-subset. New upper bounds on the sin…
$ε$-Minimum Storage Regenerating ($ε$-MSR) codes form a special class of Maximum Distance Separable (MDS) codes, providing mechanisms for exact regeneration of a single code block in their codewords by downloading slighly sub-optimal amount of information from the remaining code blocks. The key advantage of these codes is a significantly lower sub-packetization that grows only logarithmically with the length of the code, while providing optimality in storage and error-correcting capacity. However, from an implementation point of view, these codes require each remaining code block to be availab…
Given a regular multiset $M$ on $[n]=\{1,2,\ldots,n\}$, a partial order $R$ on $M$, and a label map $π: [n] \rightarrow \mathbb{N}$ defined by $π(i) = k_i$ with $\sum_{i=1}^{n}π(i) = N$, we define a pomset block metric $d_{(Pm,π)}$ on the direct sum $ \mathbb{Z}_{m}^{k_1} \oplus \mathbb{Z}_{m}^{k_2} \oplus \ldots \oplus \mathbb{Z}_{m}^{k_n}$ of $\mathbb{Z}_{m}^{N}$ based on the pomset $\mathbb{P}=(M,R)$. The pomset block metric extends the classical pomset metric introduced by I. G. Sudha and R. S. Selvaraj and generalizes the poset block metric introduced by M. M. S. Alves et al over $\mathbb…
We consider the decoding of linear and array codes from errors when we are only allowed to download a part of the codeword. More specifically, suppose that we have encoded $k$ data symbols using an $(n,k)$ code with code length $n$ and dimension $k.$ During storage, some of the codeword coordinates might be corrupted by errors. We aim to recover the original data by reading the corrupted codeword with a limit on the transmitting bandwidth, namely, we can only download an $α$ proportion of the corrupted codeword. For a given $α,$ our objective is to design a code and a decoding scheme such that…
An [n, k] linear code C that is subject to locality constraints imposed by a parity check matrix H0 is said to be a maximally recoverable (MR) code if it can recover from any erasure pattern that some k-dimensional subcode of the null space of H0 can recover from. The focus in this paper is on MR codes constrained to have all-symbol locality r. Given that it is challenging to construct MR codes having small field size, we present results in two directions. In the first, we relax the MR constraint and require only that apart from the requirement of being an optimum all-symbol locality code, the…
We construct six new explicit families of linear maximum sum-rank distance (MSRD) codes, each of which has the smallest field sizes among all known MSRD codes for some parameter regime. Using them and a previous result of the author, we provide two new explicit families of linear partial MDS (PMDS) codes with smaller field sizes than previous PMDS codes for some parameter regimes. Our approach is to characterize evaluation points that turn extended Moore matrices into the parity-check matrix of a linear MSRD code. We then produce such sequences from codes with good Hamming-metric parameters. T…
Partial maximum distance separable (PMDS) codes are a kind of erasure codes where the nodes are divided into multiple groups with each forming an MDS code with a smaller code length, thus they allow repairing a failed node with only a few helper nodes and can correct all erasure patterns that are information-theoretically correctable. However, the repair of a failed node of PMDS codes still requires a large amount of communication if the group size is large. Recently, PMDS array codes with each local code being an MSR code were introduced to reduce the repair bandwidth further. However, they r…
