Accelerating LLM Inference via Dynamic KV Cache Placement in Heterogeneous Memory System
机构 * Rensselaer Polytechnic Institute(拉特格斯理工学院) ; IBM T.J. Watson Research Center(IBM 沃森研究中心)
专题命中 长上下文与记忆 :LLM(title,abstract);large language model(abstract);language model(abstract);分类 cs.AI
Comments IEEE Computer Architecture Letter