EvolMem：一种基于认知的多会话对话记忆基准
EvolMem: A Cognitive-Driven Benchmark for Multi-Session Dialogue Memory
AI总结：
EvolMem提出了一种基于认知的多会话对话记忆基准，用于评估LLMs和智能体系统的多会话记忆能力，揭示了不同模型在多维记忆任务中的表现差异。
摘要：
尽管在理解和利用长程对话记忆方面取得了最近的进步，现有的基准测试仍然缺乏对大型语言模型（LLMs）在多种记忆维度上系统评估，特别是在多会话设置中。在本工作中，我们提出了EvolMem，一个新的基准测试，用于评估LLMs和智能体系统的多会话记忆能力。EvolMem基于认知心理学，涵盖声明性和非声明性记忆，并进一步分解为多种细粒度能力。为了构建该基准测试，我们引入了一个混合数据合成框架，包括主题引导生成和叙事启发转换。该框架能够规模化生成具有可控复杂度的多会话对话，并附带样本特定的评估指南。广泛的评估表明，没有任何LLM在所有记忆维度上都优于其他模型。此外，智能体记忆机制不一定增强LLMs的能力，往往表现出显著的效率限制。数据和代码将在https://github.com/shenye7436/EvolMem上发布。
英文摘要：
Despite recent advances in understanding and leveraging long-range conversational memory, existing benchmarks still lack systematic evaluation of large language models(LLMs) across diverse memory dimensions, particularly in multi-session settings. In this work, we propose EvolMem, a new benchmark for assessing multi-session memory capabilities of LLMs and agent systems. EvolMem is grounded in cognitive psychology and encompasses both declarative and non-declarative memory, further decomposed into multiple fine-grained abilities. To construct the benchmark, we introduce a hybrid data synthesis framework that consists of topic-initiated generation and narrative-inspired transformations. This framework enables scalable generation of multi-session conversations with controllable complexity, accompanied by sample-specific evaluation guidelines. Extensive evaluation reveals that no LLM consistently outperforms others across all memory dimensions. Moreover, agent memory mechanisms do not necessarily enhance LLMs' capabilities and often exhibit notable efficiency limitations. Data and code will be released at https://github.com/shenye7436/EvolMem.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
