当个性化合理化风险：揭示个性化对话代理中的安全漏洞
When Personalization Legitimizes Risks: Uncovering Safety Vulnerabilities in Personalized Dialogue Agents
AI总结：
本文研究了个性化对话代理中的一种安全故障模式——意图合理化，通过引入PS-Bench基准测试，揭示了个性化记忆如何偏移意图推断并导致模型合理化有害查询，提出了一种轻量级的检测-反思方法以减少安全退化。
摘要：
长期记忆使大型语言模型（LLM）代理能够支持个性化和持续的交互。然而，大多数关于个性化代理的研究优先考虑效用和用户体验，将记忆视为中性组件，并在很大程度上忽略了其安全影响。在本文中，我们揭示了意图合理化，一种此前未被充分探讨的安全故障，在个性化代理中，良性个人记忆会偏移意图推断，导致模型合理化本质上有害的查询。为了研究这一现象，我们引入了PS-Bench，一个用于识别和量化个性化交互中意图合理化的基准测试。在多个增强记忆的代理框架和基础LLM中，个性化将攻击成功率提高了15.8%至243.7%相对于无状态基线。我们进一步从内部表示空间提供了意图合理化的机理证据，并提出了一种轻量级的检测-反思方法，有效减少了安全退化。总体而言，我们的工作提供了首次系统探索和评估意图合理化作为一种安全故障模式，这种模式自然地从良性、现实世界的个性化中产生，突显了在长期个人背景下评估安全的重要性。我们的代码可在：https://github.com/MuyuenLP/PS-Bench获得。警告：本文可能包含有害内容。
英文摘要：
Long-term memory enables large language model (LLM) agents to support personalized and sustained interactions. However, most work on personalized agents prioritizes utility and user experience, treating memory as a neutral component and largely overlooking its safety implications. In this paper, we reveal intent legitimation, a previously underexplored safety failure in personalized agents, where benign personal memories bias intent inference and cause models to legitimize inherently harmful queries. To study this phenomenon, we introduce PS-Bench, a benchmark designed to identify and quantify intent legitimation in personalized interactions. Across multiple memory-augmented agent frameworks and base LLMs, personalization increases attack success rates by 15.8\%--243.7\% relative to stateless baselines. We further provide mechanistic evidence for intent legitimation from internal representations space, and propose a lightweight detection-reflection method that effectively reduces safety degradation. Overall, our work provides the first systematic exploration and evaluation of intent legitimation as a safety failure mode that naturally arises from benign, real-world personalization, highlighting the importance of assessing safety under long-term personal context. Our code is available at: https://github.com/MuyuenLP/PS-Bench. WARNING: This paper may contain harmful content.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
