勿伤人：通过基于人设的客户端模拟攻击暴露大语言模型的隐藏漏洞
Do No Harm: Exposing Hidden Vulnerabilities of LLMs via Persona-based Client Simulation Attack in Psychological Counseling
AI总结：
本文提出基于人设的客户端模拟攻击框架PCSA，通过模拟心理辅导中的客户端对话，揭示大语言模型在心理安全对齐方面的漏洞，实验表明其在七个通用和心理健康专用LLM上表现优于其他基线方法。
摘要：
随着大语言模型（LLMs）在心理健康护理中的广泛应用，高风险治疗互动的安全问题日益突出。一个关键挑战是区分治疗共情与适应不良的验证，其中支持性回应可能在多轮对话中无意强化有害信念或行为。现有红队框架主要关注通用危害或优化攻击，未能识别此类风险。为解决这一缺口，我们引入了基于人设的客户端模拟攻击（PCSA），这是首个通过连贯的人设驱动客户端对话模拟心理辅导中的客户端，以暴露LLMs在心理安全对齐方面的漏洞的红队框架。在七个通用和心理健康专用LLM上的实验表明，PCSA在四个竞争基线方法上表现显著优于。困惑度分析和人工检查进一步表明，PCSA生成的对话更自然且真实。我们的结果揭示了当前LLMs仍易受领域特定对抗性战术攻击，包括提供未经授权的医疗建议、强化妄想和隐含鼓励冒险行为。
英文摘要：
The increasing use of large language models (LLMs) in mental healthcare raises safety concerns in high-stakes therapeutic interactions. A key challenge is distinguishing therapeutic empathy from maladaptive validation, where supportive responses may inadvertently reinforce harmful beliefs or behaviors in multi-turn conversations. This risk is largely overlooked by existing red-teaming frameworks, which focus mainly on generic harms or optimization-based attacks. To address this gap, we introduce Personality-based Client Simulation Attack (PCSA), the first red-teaming framework that simulates clients in psychological counseling through coherent, persona-driven client dialogues to expose vulnerabilities in psychological safety alignment. Experiments on seven general and mental health-specialized LLMs show that PCSA substantially outperforms four competitive baselines. Perplexity analysis and human inspection further indicate that PCSA generates more natural and realistic dialogues. Our results reveal that current LLMs remain vulnerable to domain-specific adversarial tactics, providing unauthorized medical advice, reinforcing delusions, and implicitly encouraging risky actions.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
