arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2008.12707cs.ITcs.DCcs.NImath.IT
收藏

分布式存储中代码转换的带宽开销:基本限制与优化构造

Bandwidth Cost of Code Conversions in Distributed Storage: Fundamental Limits and Optimal Constructions

Francisco Maturana, K. V. Rashmi

摘要:

擦除码已经成为分布式存储系统中不可或缺的一部分,作为在设备故障的持续威胁下提供数据可靠性和持久性的工具。在这样的系统中,有限域$\mathbb{F}_q$上的$[n, k]$码将$k$消息符号编码为来自$\mathbb{F}_q$的$n$码字符号,然后将其存储在系统中的$n$不同节点上。最近的工作表明,通过调整$n$和$k$以适应设备故障率的变化,可以显着节省存储空间。这种调整需要代码转换:将初始$[n^I, k^I]$代码下的已编码数据转换为最终$[n^F, k^F]$代码下的等效数据的过程。默认的转换方法是重新编码数据,这会给系统资源带来很大的负担。可转换码是最近提出的用于实现资源高效转换的一类码。关于可转换代码的现有工作集中在最小化访问成本,即,在转换过程中访问的代码符号数。带宽对应于读取和传输的数据量,是另一个需要优化的重要资源。在本文中,我们开始研究的基本限制,在代码转换过程中使用的带宽和目前的建设,带宽最佳的可转换码。首先,我们使用具有可变容量边的网络信息流图来建模代码转换问题。其次,专注于MDS码和一个重要的参数制度称为合并制度,我们得到严格的带宽转换成本的下限。推导出的界限表明,带宽成本可以显着降低,即使在政权的接入成本不能降低相比,默认的方法。第三,我们提出了一个新的MDS可转换码的建设,它符合建议的下限,因此是带宽最佳的转换过程中。

英文摘要:

Erasure codes have become an integral part of distributed storage systems as a tool for providing data reliability and durability under the constant threat of device failures. In such systems, an $[n, k]$ code over a finite field $\mathbb{F}_q$ encodes $k$ message symbols into $n$ codeword symbols from $\mathbb{F}_q$ which are then stored on $n$ different nodes in the system. Recent work has shown that significant savings in storage space can be obtained by tuning $n$ and $k$ to variations in device failure rates. Such a tuning necessitates code conversion: the process of converting already encoded data under an initial $[n^I, k^I]$ code to its equivalent under a final $[n^F, k^F]$ code. The default approach to conversion is to reencode data, which places significant burden on system resources. Convertible codes are a recently proposed class of codes for enabling resource-efficient conversions. Existing work on convertible codes has focused on minimizing access cost, i.e., the number of code symbols accessed during conversion. Bandwidth, which corresponds to the amount of data read and transferred, is another important resource to optimize. In this paper, we initiate the study on the fundamental limits on bandwidth used during code conversion and present constructions for bandwidth-optimal convertible codes. First, we model the code conversion problem using network information flow graphs with variable capacity edges. Second, focusing on MDS codes and an important parameter regime called the merge regime, we derive tight lower bounds on the bandwidth cost of conversion. The derived bounds show that bandwidth cost can be significantly reduced even in regimes where access cost cannot be reduced as compared to the default approach. Third, we present a new construction for MDS convertible codes which matches the proposed lower bound and is thus bandwidth-optimal during conversion.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑