Understanding Is Done Early: A Depth Division of Labor in Large Language Models and Its Use for Unbounded-Context Memory
理解在早期完成:大语言模型的深度分工及其在无界上下文记忆中的应用
机构 * Tsinghua University(清华大学) ; Tencent(腾讯)
AI总结 该研究提出CoMem方法,利用大语言模型的深度分工构建无界上下文记忆,在RULER、LoCoMo等基准上性能优于全上下文KV-Direct,内存占用低、预填充速度快,证明长上下文记忆可沿层轴组织。
Comments 19 pages, 4 figures, 27 tables. Submitted to ACL Rolling Review