arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2407.18121cs.CV
收藏

基于 Elastic Cache 的视觉指令跟随模型的高效推理

Efficient Inference of Vision Instruction-Following Models with Elastic Cache

Tsinghua University(清华大学);University of Washington(华盛顿大学);Tencent(腾讯);Carnegie Mellon University(卡内基梅隆大学);Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学);Allen Institute for AI(艾伦人工智能研究所)

Zuyan Liu, Benlin Liu, Jiahui Wang, Yuhao Dong, Guangyi Chen, Yongming Rao, Ranjay Krishna, Jiwen Lu

AI总结:

本文提出 Elastic Cache 方法,通过在指令编码和输出生成阶段应用不同的加速策略,利用重要性驱动的缓存合并机制减少冗余,在提升 LVLMs 推理效率的同时显著优于现有修剪方法。

摘要:

在指令跟随大型视觉语言模型(LVLMs)领域,这些模型的高效部署面临挑战,主要由于其键值(KV)缓存的高内存需求。针对 LLMs 的传统缓存管理策略侧重于缓存驱逐,这通常无法满足多模态指令跟随模型的具体需求。认识到这一差距,在本文中,我们引入了 Elastic Cache,这是一种新颖的方法,受益于在指令编码和输出生成阶段应用不同的加速方法。我们研究了不同阶段的重要性度量,并提出了一种重要性驱动的缓存合并策略来修剪冗余缓存。我们的策略不是丢弃较不重要的缓存,而是识别重要的键/值向量作为锚点。周围的较不重要的缓存随后与这些锚点合并,增强了 KV 缓存中上下文信息的保留,同时产生任意的加速比。对于指令编码,我们利用频率来评估缓存的重要性。关于输出生成,我们基于带有偏移量的距离来优先处理 token,通过这种方式,初始和最近的 token 都被保留。在一系列 LVLMs 上的结果表明,Elastic Cache 不仅提高了效率,而且在各种任务的语言生成方面显著优于现有的修剪方法。代码可在 https://github.com/liuzuyan/ElasticCache 获取。

英文摘要:

In the field of instruction-following large vision-language models (LVLMs), the efficient deployment of these models faces challenges, notably due to the high memory demands of their key-value (KV) caches. Conventional cache management strategies for LLMs focus on cache eviction, which often fails to address the specific needs of multimodal instruction-following models. Recognizing this gap, in this paper, we introduce Elastic Cache, a novel approach that benefits from applying distinct acceleration methods for instruction encoding and output generation stages. We investigate the metrics of importance in different stages and propose an importance-driven cache merging strategy to prune redundancy caches. Instead of discarding less important caches, our strategy identifies important key/value vectors as anchor points. Surrounding less important caches are then merged with these anchors, enhancing the preservation of contextual information in the KV caches while yielding an arbitrary acceleration ratio. For instruction encoding, we utilize the frequency to evaluate the importance of caches. Regarding output generation, we prioritize tokens based on their distance with an offset, by which both the initial and most recent tokens are retained. Results on a range of LVLMs demonstrate that Elastic Cache not only boosts efficiency but also notably outperforms existing pruning methods in language generation across various tasks. Code is available at https://github.com/liuzuyan/ElasticCache

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑