记忆决定差异：评估不同记忆角色如何塑造对话代理
Memory Makes the Difference: Evaluating How Different Memory Roles Shape Conversational Agents
AI总结：
本研究提出对话记忆细粒度分类法，通过用户中心评估框架，揭示不同类型记忆（如澄清记忆、无关记忆）对RAG对话系统响应准确性、个性化及主题相关性的差异化影响。
摘要：
先前关于基于RAG的对话系统中记忆机制的研究强调了记忆如何存储和检索。然而，关于具有不同功能角色的记忆如何影响响应质量，特别是它们如何在不同对话上下文中塑造代理的响应，以及它们是否导致实质不同的响应行为，人们知之甚少。现有的对话系统评估也大多基于参考，不足以捕捉可能以不同方式满足用户偏好的响应中的细微差别。在这项工作中，我们探讨了不同记忆类型在塑造代理响应中的影响。我们提出了一个细粒度的对话记忆分类法，将检索到的记忆分类为不同的角色类型，并设计了一个模拟用户视角的用户中心评估框架。通过在长期数据集和前沿LLM上的比较实验，我们的分析揭示了记忆的许多差异化效应：例如，澄清记忆提高了响应的事实准确性和约束意识，使其更正确和个性化；无关记忆降低了主题相关性并削弱了约束意识。尽管前沿LLM功能强大，但这些发现揭示了如何利用不同记忆类型来产生更个性化的响应，并激发该方向的进一步研究。
英文摘要：
Prior research on memory mechanism in RAG-based conversational system has emphasized how memory is stored and retrieved. However, far less is known about how memories with different functional roles influence response quality. Specifically, how they shape an agent's responses under varying conversational contexts and whether they lead to substantively different response behaviors. Existing evaluations in conversational system are also largely reference-based, insufficiently capturing the nuances in responses that may address users' preferences differently. In this work, we probe the impact of different memory types in shaping agents' responses. We present a fine-grained taxonomy of conversational memory, classify retrieved memories into different role types, and design a user-centric evaluation framework that simulates user perspectives. Through comparative experiments on long-term datasets and frontier LLMs, our analysis reveal many differentiated effects of memories: e.g., clarifying memory improves responses' factual accuracy and constraint awareness, making them more correct and personalized; irrelevant memory reduces topic relevance and degrades constraint awareness. Despite the power of frontier LLMs, these findings shed light on how different memory types can be leveraged to produce more personalized responses and inspire further research in this direction.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
