arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2407.00467cs.LGcs.DCeess.IV
收藏

VcLLM:视频编解码器实为张量编解码器

VcLLM: Video Codecs are Secretly Tensor Codecs

Duke University(杜克大学);Carnegie Mellon University(卡内基梅隆大学)

Ceyu Xu, Yongji Wu, Xinyu Yang, Beidi Chen, Matthew Lentz, Danyang Zhuo, Lisa Wu Wills

AI总结:

本文提出VcLLM框架,将GPU硬件视频编解码器重新用作通用张量编解码器,在多种任务中达到最先进压缩效率,从而大幅降低大语言模型训练与推理的内存和通信带宽需求,使其可在消费级GPU上运行。

摘要:

随着大语言模型(LLMs)参数规模的持续扩大,大内存占用和高通信带宽需求已成为LLMs训练与推理的重要瓶颈。为缓解这些瓶颈,已有多种张量压缩技术被提出,以减小数据规模,从而降低内存需求和通信压力。我们的研究发现,视频编解码器尽管最初是为压缩视频而设计,但在压缩各类张量时表现出卓越的效率。我们证明,视频编解码器可以成为通用且多用途的张量编解码器,同时在多种任务中达到最先进的压缩效率。我们进一步利用GPU上可用的硬件视频编码与解码模块,构建了一个既能推理也能训练的框架,将视频编解码器重新用作张量编解码器。这大幅降低了对内存容量和通信带宽的要求,使得在消费级GPU上训练和推理大模型成为可能。

英文摘要:

As the parameter size of large language models (LLMs) continues to expand, the need for a large memory footprint and high communication bandwidth have become significant bottlenecks for the training and inference of LLMs. To mitigate these bottlenecks, various tensor compression techniques have been proposed to reduce the data size, thereby alleviating memory requirements and communication pressure. Our research found that video codecs, despite being originally designed for compressing videos, show excellent efficiency when compressing various types of tensors. We demonstrate that video codecs can be versatile and general-purpose tensor codecs while achieving the state-of-the-art compression efficiency in various tasks. We further make use of the hardware video encoding and decoding module available on GPUs to create a framework capable of both inference and training with video codecs repurposed as tensor codecs. This greatly reduces the requirement for memory capacity and communication bandwidth, enabling training and inference of large models on consumer-grade GPUs.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑