G-KV: Decoding-Time KV Cache Eviction with Global Attention
G-KV:基于全局注意力的解码时间KV缓存淘汰
Mengqi Liao, Lu Wang, Chaoyun Zhang, Zekai Shen, Xiaowei Mao, Si Qin, Qingwei Lin, Saravan Rajmohan, Dongmei Zhang, Huaiyu Wan
机构
*
School of Computer Science and Technology, Beijing Jiaotong University(计算机科学与技术学院,北京交通大学)
;
MicroSoft(微软)
;
Beijing Key Laboratory of Traffic Data Mining and Embodied Intelligence(北京交通数据挖掘与具身智能重点实验室)
专题命中
长上下文与记忆
:large language model(abstract);language model(abstract);post-training(abstract);分类 cs.CL、cs.AI
机构
*
Department of Electrical and Computer Engineering, University of Texas at Dallas(得克萨斯大学达拉斯分校电气与计算机工程系)
;
Advanced Architecture Research Group at Intel Corporation(英特尔公司先进架构研究组)
;
Department of Electrical, Computer and Systems Engineering, Rensselaer Polytechnic Institute(拉特格斯理工学院电气、计算机与系统工程系)
专题命中
长上下文与记忆
:LLM(abstract);large language model(abstract);language model(abstract);分类 cs.LG