arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2603.20586cs.LGcs.AI
收藏

MKA:用于高效长上下文推理的内存键注意

MKA: Memory-Keyed Attention for Efficient Long-Context Reasoning

University of California, Los Angeles(加州大学洛杉矶分校);Columbia University(哥伦比亚大学);University of Wisconsin-Madison(威斯康星大学麦迪逊分校)

Dong Liu, Yanxuan Yu, Ben Lengerich, Ying Nian Wu

AI总结:

本文提出MKA机制,通过多级KV缓存和动态路由提升长上下文推理效率,FastMKA在保持准确率的同时显著提升训练速度和评估效率。

摘要:

随着长上下文语言模型的重要性增加,维护和关注大型键/值(KV)缓存的成本迅速上升,成为训练和推理的主要瓶颈。尽管多查询注意(MQA)和多潜在注意(MLA)等方法通过共享或压缩KV特征来减少内存,但通常会牺牲表示质量或引入运行时开销。我们提出了内存键注意(MKA),一种层次化注意机制,整合了多级KV缓存(本地、会话和长期)并动态学习在它们之间路由注意。我们进一步引入了路由融合的MKA(FastMKA),一种广播路由变体,在注意计算前融合内存源以提高效率。在不同序列长度的实验中,FastMKA实现了良好的准确性-效率权衡:与MLA相比, perplexity相当,但训练吞吐量快5倍,评估延迟低1.8倍。这些结果突显了MKA作为高效长上下文注意的实用且可扩展的框架。

英文摘要:

As long-context language modeling becomes increasingly important, the cost of maintaining and attending to large Key/Value (KV) caches grows rapidly, becoming a major bottleneck in both training and inference. While prior works such as Multi-Query Attention (MQA) and Multi-Latent Attention (MLA) reduce memory by sharing or compressing KV features, they often trade off representation quality or incur runtime overhead. We propose Memory-Keyed Attention (MKA), a hierarchical attention mechanism that integrates multi-level KV caches (local, session, and long-term) and learns to route attention across them dynamically. We further introduce Route-Fused MKA (FastMKA), a broadcast-routed variant that fuses memory sources before attention computation for improved efficiency. Experiments on different sequence lengths show that FastMKA achieves a favorable accuracy-efficiency trade-off: comparable perplexity to MLA while achieving up to 5x faster training throughput and 1.8x lower evaluation latency. These results highlight MKA as a practical and extensible framework for efficient long-context attention.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑