Rpair：使用Rsync重新缩放RePair
Rpair: Rescaling RePair with Rsync
摘要：
数据压缩是管理大量但重复的数据集的强大工具，特别是基于语法的压缩方案，它支持对数据进行计算而不解压缩。在最好的情况下，这种方案需要一个数据集，它必须存储在磁盘上，并将其缩小到足以在内存中存储和处理。然而，即使这样，该方案本质上也是无用的，除非它可以在原始数据集上合理快速地构建，同时将数据集保留在磁盘上。在本文中，我们将展示如何使用上下文触发的分段散列来预处理这些数据集，以便之后我们可以更容易地应用RePair和其他基于语法的压缩器。我们首先给出我们的算法，然后展示如何使用它的变体来近似LZ77解析，然后利用它来证明压缩的理论界限，最后给出实验证据证明我们的方法在实践中具有竞争力。
英文摘要：
Data compression is a powerful tool for managing massive but repetitive datasets, especially schemes such as grammar-based compression that support computation over the data without decompressing it. In the best case such a scheme takes a dataset so big that it must be stored on disk and shrinks it enough that it can be stored and processed in internal memory. Even then, however, the scheme is essentially useless unless it can be built on the original dataset reasonably quickly while keeping the dataset on disk. In this paper we show how we can preprocess such datasets with context-triggered piecewise hashing such that afterwards we can apply RePair and other grammar-based compressors more easily. We first give our algorithm, then show how a variant of it can be used to approximate the LZ77 parse, then leverage that to prove theoretical bounds on compression, and finally give experimental evidence that our approach is competitive in practice.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
