Double Circulant Minimum Storage Regenerating Codes
英文摘要：
A newer version will appear soon
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
继续探索
相关论文
Information-theoretically Secure Regenerating Codes for Distributed Storage
Regenerating codes are a class of codes for distributed storage networks that provide reliability and availability of data, and also perform efficient node repair. Another important aspect of a distributed storage network is its security. In this paper, we consider a threat model where an eavesdropper may gain access to the data stored in a subset of the storage nodes, and possibly also, to the data downloaded during repair of some nodes. We provide explicit constructions of regenerating codes that achieve information-theoretic secrecy capacity in this setting.
cs.DCcs.ITcs.NI
Regenerating Codes for Errors and Erasures in Distributed Storage
Regenerating codes are a class of codes proposed for providing reliability of data and efficient repair of failed nodes in distributed storage systems. In this paper, we address the fundamental problem of handling errors and erasures during the data-reconstruction and node-repair operations. We provide explicit regenerating codes that are resilient to errors and erasures, and show that these codes are optimal with respect to storage and bandwidth requirements. As a special case, we also establish the capacity of a class of distributed storage systems in the presence of malicious adversaries. W…
cs.DCcs.ITcs.NI
Cooperative Regenerating Codes for Distributed Storage Systems
When there are multiple node failures in a distributed storage system, regenerating the failed storage nodes individually in a one-by-one manner is suboptimal as far as repair-bandwidth minimization is concerned. If data exchange among the newcomers is enabled, we can get a better tradeoff between repair bandwidth and the storage per node. An explicit and optimal construction of cooperative regenerating code is illustrated.
cs.DCcs.ITmath.IT
Exact Scalar Minimum Storage Coordinated Regenerating Codes
We study the exact and optimal repair of multiple failures in codes for distributed storage. More particularly, we examine the use of interference alignment to build exact scalar minimum storage coordinated regenerating codes (MSCR). We show that it is possible to build codes for the case of k = 2 and d > k by aligning interferences independently but that this technique cannot be applied as soon as k > 2 and d > k. Our results also apply to adaptive regenerating codes.
cs.DCcs.ITmath.IT
Exact Minimum-Repair-Bandwidth Cooperative Regenerating Codes for Distributed Storage Systems
In order to provide high data reliability, distributed storage systems disperse data with redundancy to multiple storage nodes. Regenerating codes is a new class of erasure codes to introduce redundancy for the purpose of improving the data repair performance in distributed storage. Most of the studies on regenerating codes focus on the single-failure recovery, but it is not uncommon to see two or more node failures at the same time in large storage networks. To exploit the opportunity of repairing multiple failed nodes simultaneously, a cooperative repair mechanism, in the sense that the node…
cs.DCcs.ITmath.IT
Optimal Exact-Regenerating Codes for Distributed Storage at the MSR and MBR Points via a Product-Matrix Construction
Regenerating codes are a class of distributed storage codes that optimally trade the bandwidth needed for repair of a failed node with the amount of data stored per node of the network. Minimum Storage Regenerating (MSR) codes minimize first, the amount of data stored per node, and then the repair bandwidth, while Minimum Bandwidth Regenerating (MBR) codes carry out the minimization in the reverse order. An [n, k, d] regenerating code permits the data to be recovered by connecting to any k of the n nodes in the network, while requiring that repair of a failed node be made possible by connectin…
cs.DCcs.ITcs.NI
更多相关论文收起更多论文
Interference Alignment in Regenerating Codes for Distributed Storage: Necessity and Code Constructions
Regenerating codes are a class of recently developed codes for distributed storage that, like Reed-Solomon codes, permit data recovery from any arbitrary k of n nodes. However regenerating codes possess in addition, the ability to repair a failed node by connecting to any arbitrary d nodes and downloading an amount of data that is typically far less than the size of the data file. This amount of download is termed the repair bandwidth. Minimum storage regenerating (MSR) codes are a subclass of regenerating codes that require the least amount of network storage; every such code is a maximum dis…
cs.DCcs.ITcs.NI
Simple Regenerating Codes: Network Coding for Cloud Storage
Network codes designed specifically for distributed storage systems have the potential to provide dramatically higher storage efficiency for the same availability. One main challenge in the design of such codes is the exact repair problem: if a node storing encoded information fails, in order to maintain the same level of reliability we need to create encoded information at a new node. One of the main open problems in this emerging area has been the design of simple coding schemes that allow exact and low cost repair of failed nodes and have high data rates. In particular, all prior known expl…
cs.DCcs.ITcs.NI
Secure Regenerating Codes for Reducing Storage and Bootstrap Costs in Sharded Blockchains
Blockchain is a distributed ledger with wide applications. Due to the increasing storage requirement for blockchains, the computation can be afforded by only a few miners. Sharding has been proposed to scale blockchains so that storage and transaction efficiency of the blockchain improves at the cost of security guarantee. This paper aims to consider a new protocol, Secure-Repair-Blockchain (SRB), which aims to decrease the storage cost at the miners. In addition, SRB also decreases the bootstrapping cost, which allows for new miners to easily join a sharded blockchain. In order to reduce stor…
cs.CRcs.DCcs.IT
Fast Product-Matrix Regenerating Codes
Distributed storage systems support failures of individual devices by the use of replication or erasure correcting codes. While erasure correcting codes offer a better storage efficiency than replication for similar fault tolerance, they incur higher CPU consumption, higher network consumption and higher disk I/Os. To address these issues, codes specific to storage systems have been designed. Their main feature is the ability to repair a single lost disk efficiently. In this paper, we focus on one such class of codes that minimize network consumption during repair, namely regenerating codes. W…
cs.DCcs.ITcs.PF
Exact-Regenerating Codes between MBR and MSR Points
In this paper we study distributed storage systems with exact repair. We give a construction for regenerating codes between the minimum storage regenerating (MSR) and the minimum bandwidth regenerating (MBR) points and show that in the case that the parameters n, k, and d are close to each other our constructions are close to optimal when comparing to the known capacity when only functional repair is required. We do this by showing that when the distances of the parameters n, k, and d are fixed but the actual values approach to infinity, the fraction of the performance of our codes with exact …
cs.DCcs.ITmath.IT
Quasi-cyclic Flexible Regenerating Codes
In a distributed storage environment, where the data is placed in nodes connected through a network, it is likely that one of these nodes fails. It is known that the use of erasure coding improves the fault tolerance and minimizes the redundancy added in distributed storage environments. The use of regenerating codes not only make the most of the erasure coding improvements, but also minimizes the amount of data needed to regenerate a failed node. In this paper, a new family of regenerating codes based on quasi-cyclic codes is presented. Quasi-cyclic flexible minimum storage regenerating (QCFM…
cs.DCcs.ITmath.IT
Repairing Multiple Failures with Coordinated and Adaptive Regenerating Codes
Erasure correcting codes are widely used to ensure data persistence in distributed storage systems. This paper addresses the simultaneous repair of multiple failures in such codes. We go beyond existing work (i.e., regenerating codes by Dimakis et al.) by describing (i) coordinated regenerating codes (also known as cooperative regenerating codes) which support the simultaneous repair of multiple devices, and (ii) adaptive regenerating codes which allow adapting the parameters at each repair. Similarly to regenerating codes by Dimakis et al., these codes achieve the optimal tradeoff between sto…
cs.DCcs.ITmath.IT
CORE: Augmenting Regenerating-Coding-Based Recovery for Single and Concurrent Failures in Distributed Storage Systems
Data availability is critical in distributed storage systems, especially when node failures are prevalent in real life. A key requirement is to minimize the amount of data transferred among nodes when recovering the lost or unavailable data of failed nodes. This paper explores recovery solutions based on regenerating codes, which are shown to provide fault-tolerant storage and minimum recovery bandwidth. Existing optimal regenerating codes are designed for single node failures. We build a system called CORE, which augments existing optimal regenerating codes to support a general number of fail…
cs.DC
Quorum Sensing for Regenerating Codes in Distributed Storage
Distributed storage systems with replication are well known for storing large amount of data. A large number of replication is done in order to provide reliability. This makes the system expensive. Various methods have been proposed over time to reduce the degree of replication and yet provide same level of reliability. One recently suggested scheme is of Regenerating codes, where a file is divided in to parts which are then processed by a coding mechanism and network coding to provide large number of parts. These are stored at various nodes with more than one part at each node. These codes ca…
cs.DCcs.ITmath.IT
Cooperative Repair of Multiple Node Failures in Distributed Storage Systems
Cooperative regenerating codes are designed for repairing multiple node failures in distributed storage systems. In contrast to the original repair model of regenerating codes, which are for the repair of single node failure, data exchange among the new nodes is enabled. It is known that further reduction in repair bandwidth is possible with cooperative repair. Currently in the literature, we have an explicit construction of exact-repair cooperative code achieving all parameters corresponding to the minimum-bandwidth point. We give a slightly generalized and more flexible version of this coope…
cs.DCcs.ITmath.IT
Byzantine Fault Tolerance of Regenerating Codes
Recent years have witnessed a slew of coding techniques custom designed for networked storage systems. Network coding inspired regenerating codes are the most prolifically studied among these new age storage centric codes. A lot of effort has been invested in understanding the fundamental achievable trade-offs of storage and bandwidth usage to maintain redundancy in presence of different models of failures, showcasing the efficacy of regenerating codes with respect to traditional erasure coding techniques. For practical usability in open and adversarial environments, as is typical in peer-to-p…
cs.CRcs.DC
Regenerating Codes: A System Perspective
The explosion of the amount of data stored in cloud systems calls for more efficient paradigms for redundancy. While replication is widely used to ensure data availability, erasure correcting codes provide a much better trade-off between storage and availability. Regenerating codes are good candidates for they also offer low repair costs in term of network bandwidth. While they have been proven optimal, they are difficult to understand and parameterize. In this paper we provide an analysis of regenerating codes for practitioners to grasp the various trade-offs. More specifically we make two co…
cs.DCcs.ITcs.NI
Maximum Lifetime for Data Regeneration in Wireless Sensor Networks
Robust distributed storage systems dedicated to wireless sensor networks utilize several nodes to redundantly store sensed data so that when some storage nodes fail, the sensed data can still be reconstructed. For the same level of redundancy, erasure coding based approaches are known to require less data storage space than replication methods. To maintain the same level of redundancy when one storage node fails, erasure coded data can be restored onto some other storage node by having this node download respective pieces from other live storage nodes. Previous works showed that the benefits i…
cs.DC
Distributed Storage Codes with Repair-by-Transfer and Non-achievability of Interior Points on the Storage-Bandwidth Tradeoff
Regenerating codes are a class of recently developed codes for distributed storage that, like Reed-Solomon codes, permit data recovery from any subset of k nodes within the n-node network. However, regenerating codes possess in addition, the ability to repair a failed node by connecting to an arbitrary subset of d nodes. It has been shown that for the case of functional-repair, there is a tradeoff between the amount of data stored per node and the bandwidth required to repair a failed node. A special case of functional-repair is exact-repair where the replacement node is required to store data…
cs.DCcs.ITcs.NI
LEOS-assisted Inter-GEOS Communication via Distributed-storage Coding
We consider a space communication network consisting of Geosynchronous Earth Orbit satellites (GEOSs) and Low Earth Orbit satellites (LEOSs). In case of no direct communication link between two GEOSs, the data exchange between them is through relay by the LEOSs. In particular, the source GEOS sends coded data to multiple LEOSs based on the distributed storage framework. The destination GEOS then retrieves certain amount of data from each LEOS for data reconstruction. For the GEOS-LEOS downlink, a regenerating-code-based transmission scheme is optimized to guarantee data reconstructability, whe…
cs.DCcs.ITmath.IT
A Non-MDS Erasure Code Scheme For Storage Applications
This paper investigates the use of redundancy and self repairing against node failures in distributed storage systems, using various strategies. In replication method, access to one replication node is sufficient to reconstruct a lost node, while in MDS erasure coded systems which are optimal in terms of redundancy-reliability tradeoff, a single node failure is repaired after recovering the entire stored data. Moreover, regenerating codes yield a tradeoff curve between storage capacity and repair bandwidth. The current paper aims at investigating a new storage code. Specifically, we propose a …
cs.DCcs.ITcs.NI
A Layered Architecture for Erasure-Coded Consistent Distributed Storage
Motivated by emerging applications to the edge computing paradigm, we introduce a two-layer erasure-coded fault-tolerant distributed storage system offering atomic access for read and write operations. In edge computing, clients interact with an edge-layer of servers that is geographically near; the edge-layer in turn interacts with a back-end layer of servers. The edge-layer provides low latency access and temporary storage for client operations, and uses the back-end layer for persistent storage. Our algorithm, termed Layered Data Storage (LDS) algorithm, offers several features suitable for…
cs.DCcs.ITmath.IT
An Overview of Codes Tailor-made for Better Repairability in Networked Distributed Storage Systems
The continuously increasing amount of digital data generated by today's society asks for better storage solutions. This survey looks at a new generation of coding techniques designed specifically for the needs of distributed networked storage systems, trying to reach the best compromise among storage space efficiency, fault tolerance, and maintenance overheads. Four families of codes tailor-made for distributed settings, namely - pyramid, hierarchical, regenerating and self-repairing codes - are presented at a high level, emphasizing the main ideas behind each of these codes, and discussing th…
cs.DCcs.ITmath.IT
Network Traffic Driven Storage Repair
Recently we constructed an explicit family of locally repairable and locally regenerating codes. Their existence was proven by Kamath et al. but no explicit construction was given. Our design is based on HashTag codes that can have different sub-packetization levels. In this work we emphasize the importance of having two ways to repair a node: repair only with local parity nodes or repair with both local and global parity nodes. We say that the repair strategy is network traffic driven since it is in connection with the concrete system and code parameters: the repair bandwidth of the code, the…
cs.DCcs.ITmath.IT
Secure Determinant Codes for Distributed Storage Systems
The information-theoretic secure exact-repair regenerating codes for distributed storage systems (DSSs) with parameters $(n,k=d,d,\ell)$ are studied in this paper. We consider distributed storage systems with $n$ nodes, in which the original data can be recovered from any subset of $k=d$ nodes, and the content of any node can be retrieved from those of any $d$ helper nodes. Moreover, we consider two secrecy constraints, namely, Type-I, where the message remains secure against an eavesdropper with access to the content of any subset of up to $\ell$ nodes, and Type-II, in which the message remai…
cs.CRcs.DCcs.IT
Enabling Node Repair in Any Erasure Code for Distributed Storage
Erasure codes are an efficient means of storing data across a network in comparison to data replication, as they tend to reduce the amount of data stored in the network and offer increased resilience in the presence of node failures. The codes perform poorly though, when repair of a failed node is called for, as they typically require the entire file to be downloaded to repair a failed node. A new class of erasure codes, termed as regenerating codes were recently introduced, that do much better in this respect. However, given the variety of efficient erasure codes available in the literature, …
cs.DCcs.ITcs.NI
An Empirical Study of the Repair Performance of Novel Coding Schemes for Networked Distributed Storage Systems
Erasure coding techniques are getting integrated in networked distributed storage systems as a way to provide fault-tolerance at the cost of less storage overhead than traditional replication. Redundancy is maintained over time through repair mechanisms, which may entail large network resource overheads. In recent years, several novel codes tailor-made for distributed storage have been proposed to optimize storage overhead and repair, such as Regenerating Codes that minimize the per repair traffic, or Self-Repairing Codes which minimize the number of nodes contacted per repair. Existing studie…
cs.DC
Self-repairing Homomorphic Codes for Distributed Storage Systems
Erasure codes provide a storage efficient alternative to replication based redundancy in (networked) storage systems. They however entail high communication overhead for maintenance, when some of the encoded fragments are lost and need to be replenished. Such overheads arise from the fundamental need to recreate (or keep separately) first a copy of the whole object before any individual encoded fragment can be generated and replenished. There has been recently intense interest to explore alternatives, most prominent ones being regenerating codes (RGC) and hierarchical codes (HC). We propose as…
cs.DC
