MemoryCD：LLM代理跨领域终身个性化长期上下文用户记忆基准测试
MemoryCD: Benchmarking Long-Context User Memory of LLM Agents for Lifelong Cross-Domain Personalization
AI总结：
MemoryCD通过12个多样化领域和4种个性化任务，评估14种先进LLM基模型与6种记忆方法在单域和跨域场景下的长期上下文用户记忆能力，揭示现有方法在跨领域终身个性化中的不足。
摘要：
近年来，大型语言模型（LLMs）在上下文窗口方面取得了显著进展，达到了百万token级别，但评估记忆能力的基准仍然局限于短会话的合成对话。我们引入MemoryCD，这是首个大规模、以用户为中心、跨领域的记忆基准，源自亚马逊评论数据集中长期真实世界行为。不同于现有记忆数据集依赖脚本人物生成合成用户数据，MemoryCD追踪跨年和多个领域的真实用户互动。我们构建了一个多方面的长上下文记忆评估流水线，对14种最先进的LLM基模型和6种记忆方法，在4种不同的个性化任务上12个多样化领域进行评估，以评估代理在单域和跨域设置中模拟真实用户行为的能力。我们的分析表明，现有记忆方法在各种领域中远未满足用户需求，为跨领域终身个性化评估提供了首个测试平台。
英文摘要：
Recent advancements in Large Language Models (LLMs) have expanded context windows to million-token scales, yet benchmarks for evaluating memory remain limited to short-session synthetic dialogues. We introduce \textsc{MemoryCD}, the first large-scale, user-centric, cross-domain memory benchmark derived from lifelong real-world behaviors in the Amazon Review dataset. Unlike existing memory datasets that rely on scripted personas to generate synthetic user data, \textsc{MemoryCD} tracks authentic user interactions across years and multiple domains. We construct a multi-faceted long-context memory evaluation pipeline of 14 state-of-the-art LLM base models with 6 memory method baselines on 4 distinct personalization tasks over 12 diverse domains to evaluate an agent's ability to simulate real user behaviors in both single and cross-domain settings. Our analysis reveals that existing memory methods are far from user satisfaction in various domains, offering the first testbed for cross-domain life-long personalization evaluation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
