重新审视多模态KV缓存压缩：一种基于频域的异常KV感知方法
Revisiting Multimodal KV Cache Compression: A Frequency-Domain-Guided Outlier-KV-Aware Approach
AI总结：
本文提出FlashCache，一种基于频域的异常KV感知KV缓存压缩框架，通过保留关键KV对提升解码效率并降低内存使用。
摘要：
多模态大语言模型由于多模态KV缓存随着视觉输入长度成比例增长而面临显著的推理开销。现有多模态KV缓存压缩方法大多依赖于注意力分数来减少缓存大小，这使它们与已建立的高效注意力内核（如FlashAttention）不兼容，并忽略了值向量对注意力输出的贡献。在本文中，我们从KV矩阵分布的角度重新审视多模态KV缓存压缩。首先，我们发现多模态KV矩阵的频域能量主要集中在低频部分，并通过低通滤波器提取此主要能量。进一步，我们发现移除显著偏离此主要能量的KV对会导致性能显著下降，我们将其定义为异常KV。考虑到异常KV更可能编码对推理至关重要的特征，我们提出FlashCache，一种基于频域的、异常KV感知的KV缓存压缩框架。首先，我们引入了一个异常KV识别模块，该模块在频域中建模多模态KV矩阵的主要成分，并优先保留显著偏离它的KV对。此外，动态预算分配模块被设计为适应性地确定每层KV缓存大小以保留更多的异常KV。在多个MLLMs和基准测试中的实验表明，FlashCache优于最先进的多模态KV压缩方法，实现了在KV内存使用降低80%的同时，解码速度提升达1.69倍，同时保持任务性能。
英文摘要：
Multimodal large language models suffer from substantial inference overhead since multimodal KV Cache grows proportionally with the visual input length. Existing multimodal KV Cache compression methods mostly rely on attention score to reduce cache size, which makes them are incompatible with established efficient attention kernels (e.g., FlashAttention) and ignores the contribution of value vectors to the attention output. In this work, we revisit multimodal KV Cache compression from the perspective of the KV matrices' distribution. First, we observe that frequency-domain energy of multimodal KV matrices is predominantly concentrated in low-frequency and extract this principal energy via a low-pass filter. Further, we find that removing KV pairs that deviate substantially from this principal energy leads to a pronounced performance drop, which we define as Outlier KVs. Considering Outlier KVs are more likely to encode features critical for inference, we propose FlashCache, a frequency-domain-guided, Outlier-KV-aware KV Cache compression framework. First, we introduce an Outlier KV Recognition Module that models the principal component of multimodal KV matrices in the frequency domain and preferentially retains KV pairs that significantly deviate from it. Furthermore, Dynamic Budget Allocation Module is designed to adaptively determine the per-layer KV Cache size to retain more Outlier KVs. Experiments on multiple MLLMs and benchmarks demonstrate that FlashCache outperforms state-of-the-art multimoal KV compression methods, achieving up to 1.69 times faster decoding with 80% lower KV memory usage while maintaining task performance.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
