评估多方协作对话的长周期记忆
Evaluating Long-Horizon Memory for Multi-Party Collaborative Dialogues
AI总结：
本文提出EverMemBench，首个评估多方协作对话长周期记忆的基准，揭示当前系统在多跳推理、时间推理和记忆意识方面的局限性。
摘要：
实际应用中的长期对话记忆本质上是协作的：信息由多个参与者产生，分散在不同的群体和渠道中，随着时间推移被修改，并隐含地基于角色和社会背景。然而，目前尚无一个已建立的基准来评估在接近现实部署的交互模式下的记忆能力，因为现有基准主要集中在双人或单一主题对话上。在本文中，我们介绍了EverMemBench，这是首个为长周期协作记忆设计的基准，其基于跨越一百万个标记的多方、多群体对话，包含密集的跨主题交错、时间演化的决策以及角色条件的个性。EverMemBench通过2400个问答对在三个对实际应用至关重要的维度上评估记忆系统：细粒度回忆、记忆意识和用户资料理解。我们的评估揭示了当前系统的基本限制：多跳推理在多方归属下即使有Oracle证据也会崩溃（准确率26%），时间推理在没有显式版本语义（仅限时间戳）的情况下会失败，而记忆意识受限于检索，因为基于相似性的方法会遗漏隐含相关信息。EverMemBench因此代表了LLM记忆真实评估的一个具体步骤，并成为开发下一代能够基于时间、角色和协作交互结构进行推理的LLM的基石基准。我们的基准和代码在https://github.com/EverMind-AI/EverMemBench上公开可用。
英文摘要：
Long-term conversational memory in practical LLM applications is inherently collaborative: information is produced by multiple participants, scattered across groups and channels, revised over time, and implicitly grounded in roles and social context. Yet there is currently no established benchmark that evaluates memory under interaction patterns resembling real-world deployment, as existing benchmarks largely focus on dyadic or single-topic dialogues. In this paper, we introduce EverMemBench, the first benchmark designed for long-horizon collaborative memory, built from multi-party, multi-group conversations spanning over one million tokens with dense cross-topic interleaving, temporally evolving decisions, and role-conditioned personas. EverMemBench evaluates memory systems using 2400 QA pairs across three dimensions essential for real applications: fine-grained recall, memory awareness, and user profile understanding. Our evaluation reveals fundamental limitations of current systems: multi-hop reasoning collapses under multi-party attribution even with oracle evidence (26% accuracy), temporal reasoning fails without explicit version semantics beyond timestamps, and memory awareness is bottlenecked by retrieval, as similarity-based methods miss implicitly relevant information. EverMemBench thus represents a concrete step toward realistic evaluation of LLM memory and a cornerstone benchmark for developing next-generation LLMs that reason over time, roles, and collaborative interaction structure. Our benchmark and code are publicly available at https://github.com/EverMind-AI/EverMemBench.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
