MadaKV：面向高效多模态长上下文推理的自适应模态感知KV缓存驱逐
MadaKV: Adaptive Modality-Perception KV Cache Eviction for Efficient Multimodal Long-Context Inference
AI总结：
MadaKV提出模态自适应KV缓存驱逐策略，通过模态偏好适应与分层压缩补偿，在保持高准确率的同时减少内存占用并提升解码速度1.3至1.5倍。
摘要：
本文介绍了MadaKV，一种模态自适应的键值（KV）缓存驱逐策略，旨在提升多模态大语言模型（MLLMs）在长上下文推理中的效率。在多模态场景中，注意力头对不同模态表现出不同的偏好，导致各注意力头在模态重要性上存在显著差异。传统的KV缓存驱逐方法针对单模态设置而设计，无法捕捉模态特定信息，从而产生次优性能。MadaKV通过两个关键组件解决这些挑战：模态偏好适应和分层压缩补偿。通过动态感知注意力头内的模态信息并自适应保留关键令牌，MadaKV在保持各种多模态长上下文任务高准确率的同时，显著减少了KV缓存内存占用和模型推理解码延迟（提升1.3至1.5倍）。在代表性MLLMs和MileBench基准上的大量实验证明了MadaKV相较于现有KV缓存驱逐方法的有效性。
英文摘要：
This paper introduces MadaKV, a modality-adaptive key-value (KV) cache eviction strategy designed to enhance the efficiency of multimodal large language models (MLLMs) in long-context inference. In multimodal scenarios, attention heads exhibit varying preferences for different modalities, resulting in significant disparities in modality importance across attention heads. Traditional KV cache eviction methods, which are tailored for unimodal settings, fail to capture modality-specific information, thereby yielding suboptimal performance. MadaKV addresses these challenges through two key components: modality preference adaptation and hierarchical compression compensation. By dynamically sensing modality information within attention heads and adaptively retaining critical tokens, MadaKV achieves substantial reductions in KV cache memory footprint and model inference decoding latency (1.3 to 1.5 times improvement) while maintaining high accuracy across various multimodal long-context tasks. Extensive experiments on representative MLLMs and the MileBench benchmark demonstrate the effectiveness of MadaKV compared to existing KV cache eviction methods.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
