PersistBench: 大型语言模型何时应忘记长期记忆?
PersistBench: When Should Long-Term Memories Be Forgotten by LLMs?
AI总结:
提出 PersistBench 基准,评估 LLM 长期记忆带来的跨域泄露和记忆诱导的谄媚安全风险,发现主流模型失败率高达 53% 和 97%。
摘要:
对话助手正越来越多地将长期记忆与大型语言模型(LLM)集成。这种记忆的持久性,例如用户是素食主义者,可以增强未来对话中的个性化。然而,同样的持久性也可能引入很大程度上被忽视的安全风险。因此,我们引入 PersistBench 来衡量这些安全风险的程度。我们识别出两种长期记忆特有的风险:跨域泄露,即 LLM 不恰当地从长期记忆中注入上下文;以及记忆诱导的谄媚,即存储的长期记忆暗中强化用户偏见。我们在基准上评估了 18 个前沿和开源 LLM。我们的结果显示这些 LLM 的失败率高得惊人——跨域样本的中位失败率为 53%,谄媚样本为 97%。为了解决这个问题,我们的基准鼓励在前沿对话系统中开发更稳健、更安全的长期记忆使用方式。
英文摘要:
Conversational assistants are increasingly integrating long-term memory with large language models (LLMs). This persistence of memories, e.g., the user is vegetarian, can enhance personalization in future conversations. However, the same persistence can also introduce safety risks that have been largely overlooked. Hence, we introduce PersistBench to measure the extent of these safety risks. We identify two long-term memory-specific risks: cross-domain leakage, where LLMs inappropriately inject context from the long-term memories; and memory-induced sycophancy, where stored long-term memories insidiously reinforce user biases. We evaluate 18 frontier and open-source LLMs on our benchmark. Our results reveal a surprisingly high failure rate across these LLMs - a median failure rate of 53% on cross-domain samples and 97% on sycophancy samples. To address this, our benchmark encourages the development of more robust and safer long-term memory usage in frontier conversational systems.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。