利用无损文法压缩矩阵改进矩阵向量乘法
Improving Matrix-vector Multiplication via Lossless Grammar-Compressed Matrices
摘要：
随着当今机器学习（ML）技术正在生成巨大的数据汇集，如何有效地设计其存储和操作的问题变得至关重要。在这篇文章中，我们提出了一个新的实值矩阵的无损压缩方案，实现有效的性能方面的压缩比和线性代数运算的时间。实验表明，作为一个压缩器，我们的工具是明显优于gzip，它通常是在20%的xz在压缩比方面。此外，我们的压缩格式支持与压缩表示的大小成比例的时间和空间中的矩阵向量乘法，不像gzip和xz需要完全解压缩压缩矩阵。据我们所知，我们的无损压缩器是第一个实现的时间和空间复杂度性相匹配的理论极限表示的$k$阶统计熵的输入。为了实现进一步的时间/空间减少，我们提出了一个新的列相似性得分上的列重排序算法。我们在ML矩阵的各种数据集上的实验表明，在适度的预处理时间下，我们的列重新排序可以在矩阵向量乘法期间进一步减少高达16%的峰值内存使用。最后，我们将我们的建议与最先进的压缩线性代数（CLA）方法进行比较，结果表明我们的运行速度总是至少快两倍（在多线程设置中），并且对于大多数测试数据集都实现了更好的压缩空间占用。这证实了实验证明有效的理论界限，我们展示了我们的压缩矩阵的方法。
英文摘要：
As nowadays Machine Learning (ML) techniques are generating huge data collections, the problem of how to efficiently engineer their storage and operations is becoming of paramount importance. In this article we propose a new lossless compression scheme for real-valued matrices which achieves efficient performance in terms of compression ratio and time for linear-algebra operations. Experiments show that, as a compressor, our tool is clearly superior to gzip and it is usually within 20% of xz in terms of compression ratio. In addition, our compressed format supports matrix-vector multiplications in time and space proportional to the size of the compressed representation, unlike gzip and xz that require the full decompression of the compressed matrix. To our knowledge our lossless compressor is the first one achieving time and space complexities which match the theoretical limit expressed by the $k$-th order statistical entropy of the input. To achieve further time/space reductions, we propose column-reordering algorithms hinging on a novel column-similarity score. Our experiments on various data sets of ML matrices show that, with a modest preprocessing time, our column reordering can yield a further reduction of up to 16% in the peak memory usage during matrix-vector multiplication. Finally, we compare our proposal against the state-of-the-art Compressed Linear Algebra (CLA) approach showing that ours runs always at least twice faster (in a multi-thread setting) and achieves better compressed space occupancy for most of the tested data sets. This experimentally confirms the provably effective theoretical bounds we show for our compressed-matrix approach.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
