In this paper, we introduce a model of a single-failure locally recoverable distributed storage system. This model appears to give rise to a problem seemingly dual of the well-studied index coding problem. The relation between the dimensions of an optimal index code and optimal distributed storage code of our model has been established in this paper. We also show some extensions to vector codes.
Locally recoverable codes are widely used in distributed and cloud storage systems. The objective of this paper is to present a construction of near MDS codes with oval polynomials and then determine the locality of the codes. It turns out that the near MDS codes and their duals are both distance-optimal and dimension-optimal locally recoverable codes. The lengths of the locally recoverable codes are different from known ones in the literature.
We consider a coded compressed sensing approach for the unsourced random access and replace the outer tree code proposed by Amalladinne et al. with the list recoverable code capable of correcting t errors. A finite-length random coding bound for such codes is derived. The numerical experiments in the single antenna quasi-static Rayleigh fading MAC show that transition to list recoverable codes correcting t errors improves the performance of coded compressed sensing scheme by 7-10 dB compared to the tree code-based scheme. We propose two practical constructions of outer codes. The first is a mo…
In recent years, the rapidly increasing amounts of data created and processed through the internet resulted in distributed storage systems employing erasure coding based schemes. Aiming to balance the tradeoff between data recovery for correlated failures and efficient encoding and decoding, distributed storage systems employing maximally recoverable codes came up. Unifying a number of topologies considered both in theory and practice, Gopalan et al. \cite{Gopalan2017} initiated the study of maximally recoverable codes for grid-like topologies. In this paper, we focus on the maximally recovera…
A linear code with parameters $[n,k,n-k]$ is said to be almost maximum distance separable (AMDS for short). An AMDS code whose dual is also AMDS is referred to as an near maximum distance separable (NMDS for short) code. NMDS codes have nice applications in finite geometry, combinatorics, cryptography and data storage. In this paper, we first present several constructions of NMDS codes and determine their weight enumerators. In particular, some constructions produce NMDS codes with the same parameters but different weight enumerators. Then we determine the locality of the NMDS codes and obtain…
Motivated by the established notion of storage codes, we consider sets of infinite sequences over a finite alphabet such that every $k$-tuple of consecutive entries is uniquely recoverable from its $l$-neighborhood in the sequence. We address the problem of finding the maximum growth rate of the set, which we term capacity, as well as constructions of explicit families that approach the optimal rate. The techniques that we employ rely on the connection of this problem with constrained systems. In the second part of the paper we consider a modification of the problem wherein the entries in the …
In the emerging field of DNA storage, data is encoded as DNA sequences and stored. The data is read out again by sequencing the stored DNA. Nanopore sequencing is a new sequencing technology that has many advantages over other methods; in particular, it is cheap, portable, and can support longer reads. While several practical coding schemes have been developed for DNA storage with nanopore sequencing, the theory is not well understood. Towards that end, we study a highly abstracted (deterministic) version of the nanopore sequencer, which highlights key features that make its analysis difficult…
Error-correcting codes over sets, with applications to DNA storage, are studied. The DNA-storage channel receives a set of sequences, and produces a corrupted version of the set, including sequence loss, symbol substitution, symbol insertion/deletion, and limited-magnitude errors in symbols. Various parameter regimes are studied. New bounds on code parameters are provided, which improve upon known bounds. New codes are constructed, at times matching the bounds up to lower-or der terms or small constant factors.
Due to the redundant nature of DNA synthesis and sequencing technologies, a basic model for a DNA storage system is a multi-draw "shuffling-sampling" channel. In this model, a random number of noisy copies of each sequence is observed at the channel output. Recent works have characterized the capacity of such a DNA storage channel under different noise and sequencing models, relying on sophisticated typicality-based approaches for the achievability. Here, we consider a multi-draw DNA storage channel in the setting of noise corruption by a binary erasure channel. We show that, in this setting, …
From currency to cloud storage systems, the continuous rise of the blockchain technology is moving various information systems towards decentralization. Blockchain-based decentralized storage networks (DSNs) offer significantly higher privacy and lower costs to customers compared with centralized cloud storage associated with specific vendors. Coding is required in order to retrieve data stored on failing components. While coding solutions for centralized storage have been intensely studied, topology-aware coding for heterogeneous DSNs have not yet been discussed. In this paper, we propose a j…
In this paper we study error-correcting codes for the storage of data in synthetic deoxyribonucleic acid (DNA). We investigate a storage model where a data set is represented by an unordered set of $M$ sequences, each of length $L$. Errors within that model are a loss of whole sequences and point errors inside the sequences, such as insertions, deletions and substitutions. We derive Gilbert-Varshamov lower bounds and sphere packing upper bounds on achievable cardinalities of error-correcting codes within this storage model. We further propose explicit code constructions than can correct errors…
Motivated by applications in distributed storage, the storage capacity of a graph was recently defined to be the maximum amount of information that can be stored across the vertices of a graph such that the information at any vertex can be recovered from the information stored at the neighboring vertices. Computing the storage capacity is a fundamental problem in network coding and is related, or equivalent, to some well-studied problems such as index coding with side information and generalized guessing games. In this paper, we consider storage capacity as a natural information-theoretic anal…
In order to accommodate the ever-growing data from various, possibly independent, sources and the dynamic nature of data usage rates in practical applications, modern cloud data storage systems are required to be scalable, flexible, and heterogeneous. The recent rise of the blockchain technology is also moving various information systems towards decentralization to achieve high privacy at low costs. While codes with hierarchical locality have been intensively studied in the context of centralized cloud storage due to their effectiveness in reducing the average reading time, those for decentral…
Local List Recovery of High-rate Tensor Codes and Applications
在这项工作中，我们给出了第一个高速率的本地列表可恢复代码的建设。列表恢复一直是编码理论中非常有用的构建块，我们的动机是使用这些代码作为这样的构建块。特别是，我们的建设给出了第一个容量实现本地列表可解码的代码（在恒定大小的字母表）;第一个容量实现全局列表可解码的代码与近线性时间列表解码算法（再一次，在恒定大小的字母表）;和Gilbert-Varshamov界上的二进制码的随机化结构，可以在近线性时间内唯一解码，具有比以前已知的更高的速率。我们的技术实际上非常简单，并且受到Gopalan，Guruswami和Raghavendra（Siam Journal on Computing，2011）用于列表解码张量代码的方法的启发。我们表明，张量的权力（全球）列表可恢复的代码是“近似”本地列表可恢复的，和“近似”的修改器可以通过预编码的消息与合适的本地可解码的代码被删除。用高速率全局列表可恢复代码和高速率局部可解码代码的已知构造来实例化该构造完成该构造。
We consider the problem of storing data in a distributed manner over $T$ servers. We require the data (i) to be recoverable from the $T$ servers, and (ii) to remain private from any $T-1$ colluding servers, where privacy is quantified in terms of mutual information between the data and all the information available at the $T-1$ colluding servers. For this model, we determine (i) the fundamental trade-off between storage size and the level of desired privacy, (ii) the optimal amount of local randomness necessary at the encoder, and (iii)~an explicit low-complexity coding scheme that solely reli…
A method for encoding information in DNA sequences is described. The method is based on the precision-resolution framework, and is aimed to work in conjunction with a recently suggested terminator-free template independent DNA synthesis method. The suggested method optimizes the amount of information bits per synthesis time unit, namely, the writing rate. Additionally, the encoding scheme studied here takes into account the existence of multiple copies of the DNA sequence, which are independently distorted. Finally, quantizers for various run-length distributions are designed.
Beyond Polynomials: Optimal Locally Recoverable Codes from Good Rational Functions
Southwest Jiaotong University（西南交通大学）；University of Paris VIII（巴黎第八大学）；CNRS, UMR 7539 LAGA, University of Paris XIII（法国国家科学研究中心，UMR 7539 LAGA，巴黎第十三大学）；China West Normal University（西华师范大学）
In order to accommodate the ever-growing data from various, possibly independent, sources and the dynamic nature of data usage rates in practical applications, modern cloud data storage systems are required to be scalable, flexible, and heterogeneous. Codes with hierarchical locality have been intensively studied due to their effectiveness in reducing the average reading time in cloud storage. In this paper, we present the first codes with hierarchical locality that achieve scalability and flexibility in heterogeneous cloud storage using small field size. We propose a double-level construction…
In this paper, we introduce a model of a distributed storage system that is locally recoverable from any single server failure. Unlike the usual local recovery model of codes for distributed storage, this model accounts for the fact that each server or storage node in a network is connectible to only some, and not all other, nodes. This may happen for reasons such as physical separation, inhomogeneity in storage platforms etc. We estimate the storage capacity of both undirected and directed networks under this model and propose some constructive schemes. From a coding theory point of view, we …
Maximally recoverable codes with locality and availability
IMUVa-Mathematics Research Institute, University of Valladolid（瓦拉多利德大学数学研究所）；Signal Processing and Communications Research Center (SPCRC), International Institute of Information Technology, Hyderabad（海得拉巴国际信息学院信号处理与通信研究中心）
In this paper, we study three applications of recursion to problems in coding and random permutations. First, we consider locally recoverable codes with partial locality and use recursion to estimate the minimum distance of such codes. Next we consider weighted lattice representative codes and use recursive subadditive techniques to obtain convergence of the minimum code size. Finally, we obtain a recursive relation involving cycle moments in random permutations and as an illustration, evaluate recursions for the mean and variance.
A linear error correcting code is a subspace of a finite-dimensional space over a finite field with a fixed coordinate system. Such a code is said to be locally recoverable with locality $r$ if, for every coordinate, its value at a codeword can be deduced from the value of (certain) $r$ other coordinates of the codeword. These codes have found many recent applications, e.g., to distributed cloud storage. We will discuss the problem of constructing good locally recoverable codes and present some constructions using algebraic surfaces that improve previous constructions and sometimes provide cod…
The process of DNA-based data storage (DNA storage for short) can be mathematically modelled as a communication channel, termed DNA storage channel, whose inputs and outputs are sets of unordered sequences. To design error correcting codes for DNA storage channel, a new metric, termed the sequence-subset distance, is introduced, which generalizes the Hamming distance to a distance function defined between any two sets of unordered vectors and helps to establish a uniform framework to design error correcting codes for DNA storage channel. We further introduce a family of error correcting codes,…
In this Macaulay2 \cite{M2} package we define an object called {\it linear code}. We implement functions that compute basic parameters and objects associated with a linear code, such as generator and parity check matrices, the dual code, length, dimension, and minimum distance, among others. We define an object {\it evaluation code}, a construction which allows to study linear codes using tools of algebraic geometry and commutative algebra. We implement functions to generate important families of linear codes such as Hamming codes, cyclic codes, Reed--Solomon codes, Reed--Muller codes, Cartesi…
