Locomo-Plus：面向LLM代理的超越事实性认知记忆评估框架
Locomo-Plus: Beyond-Factual Cognitive Memory Evaluation Framework for LLM Agents
AI总结：
LoCoMo-Plus提出一种基于约束一致性的评估框架，用于评估LLM代理在长期对话中处理隐含约束的能力，揭示现有基准无法捕捉的认知记忆挑战。
摘要：
长期对话记忆是基于LLM的对话系统的核心能力，但现有基准和评估协议主要聚焦于表层事实回忆。在现实交互中，合适的回应往往依赖于隐含约束如用户状态、目标或价值观，这些并非后来显式查询。为评估此场景，我们引入LoCoMo-Plus，一个用于评估认知记忆在提示-触发语义断开情况下的基准，其中模型必须在长期对话上下文中保留并应用隐含约束。我们进一步表明，传统字符串匹配指标和显式任务类型提示与此类场景不一致，并提出基于约束一致性的统一评估框架。在多样后端模型、检索方法和记忆系统上的实验表明，认知记忆仍然具有挑战性，并揭示了现有基准无法捕捉的失败案例。我们的代码和评估框架可在https://github.com/xjtuleeyf/Locomo-Plus公开获取。
英文摘要：
Long-term conversational memory is a core capability for LLM-based dialogue systems, yet existing benchmarks and evaluation protocols primarily focus on surface-level factual recall. In realistic interactions, appropriate responses often depend on implicit constraints such as user state, goals, or values that are not explicitly queried later. To evaluate this setting, we introduce \textbf{LoCoMo-Plus}, a benchmark for assessing cognitive memory under cue--trigger semantic disconnect, where models must retain and apply latent constraints across long conversational contexts. We further show that conventional string-matching metrics and explicit task-type prompting are misaligned with such scenarios, and propose a unified evaluation framework based on constraint consistency. Experiments across diverse backbone models, retrieval-based methods, and memory systems demonstrate that cognitive memory remains challenging and reveals failures not captured by existing benchmarks. Our code and evaluation framework are publicly available at: https://github.com/xjtuleeyf/Locomo-Plus.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
