arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2602.13967cs.AIcs.CL
收藏

Neuromem: 外存中LLMs流式生命周期的粒度分解

Neuromem: A Granular Decomposition of the Streaming Lifecycle in External Memory for LLMs

Grid Computing Lab, School of Computer Science(计算机科学学院网格计算实验室)

Ruicheng Zhang, Xinyi Li, Tianyi Xu, Shuhao Zhang, Xiaofei Liao, Hai Jin

AI总结:

Neuromem通过流式生命周期分解评估外存模块,揭示内存结构和压缩策略对性能的影响,发现内存增长导致性能下降,时间查询最具挑战性。

摘要:

大多数对外存模块的评估都假设静态环境:内存是在线下构建的,并在固定状态下进行查询。实际上,内存是流式的:新的事实不断到来,插入操作与检索操作交错进行,且在模型处理查询时,内存状态会持续演变。在这种情况下,准确性和成本由完整的内存生命周期决定,这包括信息的摄入、维护、检索以及整合到生成过程中的各个方面。我们提出了Neuromem,一个可扩展的测试平台,用于在交错的插入-检索协议下评估外存模块,并将其生命周期分解为五个维度,包括内存数据结构、归一化策略、巩固策略、查询制定策略以及上下文整合机制。使用三个代表性数据集LOCOMO、LONGMEMEVAL和MEMORYAGENTBENCH,Neuromem在共享服务堆栈中评估可替换的变体,报告token级别的F1分数以及插入/检索延迟。总体而言,我们发现性能通常随着内存增长而下降,时间相关查询仍然是最具有挑战性的类别。内存数据结构在决定可达到的质量前沿方面起着重要作用,而激进的压缩和生成整合机制大多在插入和检索之间转移成本,但对准确性提升有限。

英文摘要:

Most evaluations of External Memory Module assume a static setting: memory is built offline and queried at a fixed state. In practice, memory is streaming: new facts arrive continuously, insertions interleave with retrievals, and the memory state evolves while the model is serving queries. In this regime, accuracy and cost are governed by the full memory lifecycle, which encompasses the ingestion, maintenance, retrieval, and integration of information into generation. We present Neuromem, a scalable testbed that benchmarks External Memory Modules under an interleaved insertion-and-retrieval protocol and decomposes its lifecycle into five dimensions including memory data structure, normalization strategy, consolidation policy, query formulation strategy, and context integration mechanism. Using three representative datasets LOCOMO, LONGMEMEVAL, and MEMORYAGENTBENCH, Neuromem evaluates interchangeable variants within a shared serving stack, reporting token-level F1 and insertion/retrieval latency. Overall, we observe that performance typically degrades as memory grows across rounds, and time-related queries remain the most challenging category. The memory data structure largely determines the attainable quality frontier, while aggressive compression and generative integration mechanisms mostly shift cost between insertion and retrieval with limited accuracy gain.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑