arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2407.20266cs.LGcs.CL
收藏

加速低秩分解模型

Accelerating the Low-Rank Decomposed Models

Huawei Technologies(华为技术有限公司)

Habib Hajimolahoseini, Walid Ahmed, Austin Wen, Yang Liu

AI总结:

本文研究如何改进AI模型中的低秩分解技术,以在保持高精度的同时降低内存消耗并加速训练与推理。

摘要:

张量分解是一种有数学支持的数据压缩技术。它涉及在张量或矩阵上应用某种低秩分解技术,以减少数据的冗余。然而,由于分解后架构中新增了大量层,它并不是压缩AI模型的流行技术。尽管参数数量可能显著减少,但这可能导致模型深度增加一倍以上,从而给训练或推理增加延迟。在本文中,我们提出了一项关于如何修改AI模型中低秩分解技术的全面研究,以便我们能够同时受益于高精度、低内存消耗以及加速训练和推理。

英文摘要:

Tensor decomposition is a mathematically supported technique for data compression. It consists of applying some kind of a Low Rank Decomposition technique on the tensors or matrices in order to reduce the redundancy of the data. However, it is not a popular technique for compressing the AI models duo to the high number of new layers added to the architecture after decomposition. Although the number of parameters could shrink significantly, it could result in the model be more than twice deeper which could add some latency to the training or inference. In this paper, we present a comprehensive study about how to modify low rank decomposition technique in AI models so that we could benefit from both high accuracy and low memory consumption as well as speeding up the training and inference

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑