arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~

Consolidator:学习跨上下文边界的持久路由记忆

Consolidator: Learning Persistent Routed Memory Across Context Boundaries

Sungwoo Goo, Hwi-yeol Yun, Sangkeun Jung

arXiv 2608.11701首次发表:更新:

发表机构

College of Pharmacy, Chungnam National University; Department of Computer Science & Engineering, Chungnam National University(忠南国立大学药学院; 忠南国立大学计算机科学与工程学院)

机构由 AI 辅助整理,请以论文原文为准。

AI 中文总结

该研究提出Consolidator算子,在冻结骨干与记忆接口的情况下,通过少量可训练参数实现跨上下文边界的持久路由记忆,在两段式模10映射任务中显著提升了更新后映射的召回率。

AI 中文摘要

将短期记忆(STM)复制到较慢的存储中可以在上下文边界处保留状态,但仅持久性不足以确保保留的状态会影响后续的记忆访问。我们在Phasor Memory Network(PMNet)中测试了这种区分,使用Consolidator(一种共享的槽位局部算子)在将路由的STM积累到长期记忆(LTM)之前对其进行转换,无需重放源token。每次整合后,键值(KV)缓存和STM都会被清除。保留的LTM仍可被读取,同时会被输入到分层路由器中,从而调节后续输入访问哪些显式记忆槽位。我们在一个两段式模10映射任务上评估该机制,其中第二段会更新同一记忆地址处的映射。在第二次整合和重置后,需用保留的查询从LTM中恢复更新后的映射。骨干网络和记忆接口被冻结,仅保留12.35K个Consolidator参数可训练(占29.95M模型的0.041%)。在来自同一STM预训练检查点的五组配对运行中,直接LTM路由将更新后映射的召回率从44.38±1.94%提升至87.02±1.76%(提升42.64±1.10个百分点),而两种情况下的即时STM召回率均保持89.90%;两种情况均训练独立的Consolidator并保留相同的LTM读取路径。在无路由时,学习型整合比强制恒等积累表现好21.40±1.91个百分点,在有路由时则好68.70±1.76个百分点。因此,在该任务上,整合后的LTM既作为可检索内容,又作为塑造后续槽位选择的访问状态。

英文摘要

Copying short-term memory (STM) into a slower store can preserve state across a context boundary, but persistence alone does not ensure that the retained state influences subsequent memory access. We test this distinction in a Phasor Memory Network (PMNet) using Consolidator, a shared slot-local operator that transforms routed STM before accumulating it into long-term memory (LTM), without replaying the source tokens. After each consolidation, the KV cache and STM are cleared. The retained LTM can still be read and is also fed into the hierarchical router, thereby conditioning which explicit-memory slots subsequent inputs access. We evaluate this mechanism on a two-segment modulo-10 mapping task in which the second segment updates the mapping at the same memory address. Following a second consolidation and reset, a held-out query must recover the updated mapping from LTM. The backbone and memory interface are frozen, leaving only 12.35K Consolidator parameters trainable (0.041\% of a 29.95M model). Across five paired runs from the same STM-pretraining checkpoint, direct LTM routing raises updated-mapping recall from $44.38\pm1.94\%$ to $87.02\pm1.76\%$ ($+42.64\pm1.10$ percentage points), while immediate STM recall remains 89.90\% in both conditions; both train separate Consolidators and retain the same LTM read paths. Learned consolidation outperforms forced identity accumulation by $21.40\pm1.91$ percentage points without routing and $68.70\pm1.76$ with routing. Thus, on this task, consolidated LTM serves as both retrievable content and an access state that shapes subsequent slot selection.

论文原文

arXiv 摘要页 · PDF 原文 · HTML 原文

↑