压缩卷积注意力:在压缩潜在空间中的高效注意力
Compressed Convolutional Attention: Efficient Attention in a Compressed Latent Space
AI总结:
本文提出压缩卷积注意力(CCA)和压缩卷积分组查询注意力(CCGQA),通过在共享潜在空间内执行注意力操作,显著减少参数、KV缓存和FLOPs,提升训练和推理效率。
摘要:
多头注意力(MHA)的二次计算和线性增长的KV缓存使长上下文Transformer训练和服务成本高昂。先前工作如分组查询注意力(GQA)和多潜在注意力(MLA)缩小了缓存,加速解码,但计算量仍保持不变。我们引入压缩卷积注意力(CCA),一种新的注意力方法,通过下投影查询、键和值并在共享潜在空间内执行整个注意力操作。这种简单设计通过期望的压缩因子大幅减少参数、KV缓存和FLOPs。由于CCA与头共享正交,我们结合两者形成压缩卷积分组查询注意力(CCGQA),进一步收紧计算带宽帕累托前沿,使用户可以在不牺牲质量的情况下调整压缩以适应FLOP或内存限制。实验表明,CCGQA在相同KV缓存压缩下,优于GQA和MLA,在密集和MoE模型中表现一致。此外,我们显示在具有GQA和MLA一半KV缓存的MoE模型中,CCGQA表现优于所有其他注意力方法,实现8倍的KV缓存压缩,性能无下降。CCA和CCGQA还大幅降低了注意力的FLOP成本,导致训练和预填充速度显著快于现有方法。在H100 GPU上,融合的CCA/CCGQA内核将16k序列长度的预填充延迟降低了约1.7倍,比MHA快1.3倍。
英文摘要:
Multi-headed Attention's (MHA) quadratic compute and linearly growing KV-cache make long-context transformers expensive to train and serve. Prior works such as Grouped Query Attention (GQA) and Multi-Latent Attention (MLA) shrink the cache, speeding decode, but leave compute, which determines prefill and training speed, largely unchanged. We introduce Compressed Convolutional Attention (CCA), a novel attention method which down-projects queries, keys, and values and performs the entire attention operation inside the shared latent space. This simple design dramatically cuts parameters, KV-cache, and FLOPs all at once by the desired compression factor. Because CCA is orthogonal to head-sharing, we combine the two to form Compressed Convolutional Grouped Query Attention (CCGQA), which further tightens the compute-bandwidth Pareto frontier so that users can tune compression toward either FLOP or memory limits without sacrificing quality. Experiments show that CCGQA consistently outperforms both GQA and MLA at equal KV-cache compression on dense and MoE models. Additionally, we show that CCGQA outperforms all other attention methods on MoE models with half the KV-cache of GQA and MLA, achieving an 8x KV-cache compression with no drop in performance compared to standard MHA. CCA and CCGQA also dramatically reduce the FLOP cost of attention which leads to substantially faster training and prefill than existing methods. On H100 GPUs, our fused CCA/CCGQA kernel reduces prefill latency by about 1.7x at a sequence length of 16k relative to MHA, and accelerates backward by about 1.3x.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。