发表机构
City University of Hong Kong; Huawei Noah’s Ark Lab（香港城市大学; 华为诺亚方舟实验室）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
该研究针对现有LLM记忆方法采用统一策略导致性能不足的问题，提出TriMEM数据集与MemoType框架，通过记忆与查询路由实现定制化检索，在三个数据集上的Recall@1指标最高提升16.18%。
AI 中文摘要
近年来，大语言模型（LLMs）的记忆能力受到越来越多的关注。尽管已取得巨大成功，现有的基于检索的记忆方法通常忽略记忆之间的差异，采用统一策略处理所有记忆，导致性能未达最优。因此，一个直观的问题随之产生：我们能否将记忆划分为不同类型并选择合适的策略？然而，鉴于记忆场景主题丰富、场景复杂且边界模糊，实现记忆的精确分类并非易事。为应对这一挑战，本文提出一个记忆多分类数据集，命名为TriMEM，该数据集为不同场景下的记忆类型提供精确标注。在此基础上，我们提出一种名为MemoType的新型记忆框架，该框架可通过学习到的路由模型自适应识别每个记忆和查询类型。借助记忆与查询路由，MemoType可检索对应查询类型的记忆并设计定制化检索策略，从而提升检索性能。此外，我们从理论上证明，在多类语料库中，任何单一检索策略的预期检索精度都存在基本上限，会导致系统性的精度下降。在三个数据集上开展的大量实验表明，MemoType的性能始终优于现有方法，在Recall@1指标上实现了最高16.18%的提升。
英文摘要
The memory capabilities of Large Language Models (LLMs) have garnered increasing attention recently. Despite great success achieved, existing retrieval-based memory approaches typically overlook the differences between memories and employ a unified strategy to process all memories, leading to suboptimal performance. Thus, an intuitive question arises: can we categorize memory into different types and select appropriate strategies? However, given the topic-rich, scenario-complex, and boundary-blurred nature of memory scenarios, achieving precise classification of memories is not easy. To address this challenge, we propose a memory multi-class dataset in this paper, termed TriMEM, which provides precise annotations for memory types across diverse scenarios. Building upon this foundation, we propose a novel memory framework, named MemoType, which can adaptively recognize each memory and query type with the learned router model. With the memory and query routing, MemoType can retrieve the memory with corresponding query types and design tailored retrieval strategies, thereby enhancing the retrieval performance. Moreover, we theoretically prove that any single retrieval strategy is subject to a fundamental upper bound on its expected retrieval precision in multi-class corpora, leading to systematic precision degradation. Extensive experiments on three datasets demonstrate that MemoType consistently outperforms existing methods, achieving up to 16.18% improvement in Recall@1.
CommentsNeurIPS 2026 Accept Paper
