arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2410.20650cs.LGcs.AIstat.ML
收藏

NeuZip:通过神经网络动态压缩实现内存高效的训练与推理

NeuZip: Memory-Efficient Training and Inference with Dynamic Compression of Neural Networks

University of Alberta(阿尔伯塔大学);Borealis AI(博雷亚利斯人工智能公司)

Yongchang Hao, Yanshuai Cao, Lili Mou

AI总结:

提出基于浮点数熵的权重压缩方案 NeuZip,在不损失性能的前提下显著降低神经网络训练与推理的内存占用。

摘要:

当使用更多参数时,神经网络的性能会提升。然而,在训练和推理期间,模型大小受到设备上可用内存的限制。尽管应用量化等技术可以缓解这一限制,但它们会遭遇性能下降。在本工作中,我们提出 NeuZip,这是一种基于神经网络中浮点数熵的新型权重压缩方案。借助 NeuZip,我们能够在不牺牲性能的情况下实现内存高效的训练和推理。值得注意的是,我们将训练 Llama-3 8B 模型的内存占用从 31GB 显著降低到不足 16GB,同时保持训练动态完全不变。在推理中,我们的方法可将内存使用量减少一半以上,并保持近乎无损的性能。我们的代码已公开发布。

英文摘要:

The performance of neural networks improves when more parameters are used. However, the model sizes are constrained by the available on-device memory during training and inference. Although applying techniques like quantization can alleviate the constraint, they suffer from performance degradation. In this work, we introduce NeuZip, a new weight compression scheme based on the entropy of floating-point numbers in neural networks. With NeuZip, we are able to achieve memory-efficient training and inference without sacrificing performance. Notably, we significantly reduce the memory footprint of training a Llama-3 8B model from 31GB to less than 16GB, while keeping the training dynamics fully unchanged. In inference, our method can reduce memory usage by more than half while maintaining near-lossless performance. Our code is publicly available.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑