个性化记忆如何塑造大语言模型行为？个性化助手中的理性偏好利用基准测试
How Does Personalized Memory Shape LLM Behavior? Benchmarking Rational Preference Utilization in Personalized Assistants
AI总结：
本研究提出RPEval基准，通过分析个性化记忆对LLM行为的影响，引入RP-Reasoner方法，有效缓解非理性个性化问题。
摘要：
基于大语言模型（LLM）的助手近年来集成了记录用户偏好的记忆机制，从而产生更加个性化和与用户一致的响应。然而，无关的个性化记忆经常被引入上下文，干扰LLM的意图理解。为了全面研究个性化的影响，我们开发了RPEval，一个包含个性化意图推理数据集和多粒度评估协议的基准。RPEval揭示了现有LLM中普遍存在非理性个性化现象，并通过错误模式分析展示了其对用户体验的负面影响。最后，我们引入了RP-Reasoner，将记忆利用视为一种实践推理过程，使个性化信息能够被选择性地整合。实验结果表明，我们的方法在RPEval上显著优于精心设计的基线，并解决了大规模商业个性化助手中观察到的80%的不良案例，突显了实践推理在缓解非理性个性化方面的潜力。我们的基准已公开在https://github.com/XueyangFeng/RPEval。
英文摘要：
Large language model (LLM)-powered assistants have recently integrated memory mechanisms that record user preferences, leading to more personalized and user-aligned responses. However, irrelevant personalized memories are often introduced into the context, interfering with the LLM's intent understanding. To comprehensively investigate the dual effects of personalization, we develop RPEval, a benchmark comprising a personalized intent reasoning dataset and a multi-granularity evaluation protocol. RPEval reveals the widespread phenomenon of irrational personalization in existing LLMs and, through error pattern analysis, illustrates its negative impact on user experience. Finally, we introduce RP-Reasoner, which treats memory utilization as a pragmatic reasoning process, enabling the selective integration of personalized information. Experimental results demonstrate that our method significantly outperforms carefully designed baselines on RPEval, and resolves 80% of the bad cases observed in a large-scale commercial personalized assistant, highlighting the potential of pragmatic reasoning to mitigate irrational personalization. Our benchmark is publicly available at https://github.com/XueyangFeng/RPEval.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
