CURing 大型模型:基于 CUR 分解的压缩
CURing Large Models: Compression via CUR Decomposition
AI总结:
本文提出基于 CUR 矩阵分解的模型压缩方法 CURing,结合权重大小与激活值筛选行列,在极小性能损失下快速缩减大模型参数。
摘要:
大型深度学习模型已取得显著成功,但资源消耗很高,带来了内存占用等挑战。我们提出 CURing,这是一种基于 CUR 矩阵分解的新型模型压缩方法,它将权重矩阵近似为所选列(C)和所选行(R)以及一个小型连接矩阵(U)的乘积。我们根据权重大小和激活值的综合影响,将该分解应用于所选权重。通过识别并保留信息丰富的行和列,CURing 以最小的性能损失显著减小了模型规模。例如,它仅用 129 秒就将 Llama3.1-8B 的参数量减少到 7.32B(-9%),速度比以往压缩方法快 20 多倍。
英文摘要:
Large deep learning models have achieved remarkable success but are resource-intensive, posing challenges such as memory usage. We introduce CURing, a novel model compression method based on CUR matrix decomposition, which approximates weight matrices as the product of selected columns (C) and rows (R), and a small linking matrix (U). We apply this decomposition to weights chosen based on the combined influence of their magnitudes and activations. By identifying and retaining informative rows and columns, CURing significantly reduces model size with minimal performance loss. For example, it reduces Llama3.1-8B's parameters to 7.32B (-9%) in just 129 seconds, over 20 times faster than prior compression methods.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。