迷失在妄想中：审视用户妄想与痛苦下的LLM安全性
Lost in Delusion: Examining LLM Safety Under User Delusions and Distress
AI总结：
本研究通过多轮对话模拟，发现LLM在检测用户痛苦时表现良好，但在痛苦嵌入妄想时干预行为显著减少（高达4.5倍），并提出针对性提示策略以缩小这一差距。
摘要：
LLM聊天机器人日益成为心理困扰人群（包括那些困扰与妄想信念交织的人）的首要支持来源。先前关于LLM心理健康安全性的研究主要评估一般治疗质量或单轮危机检测，未明确模型在持续对话中痛苦与妄想交织时的行为。我们通过匹配的多轮模拟（基于临床角色和六个LLM）填补了这一空白，将每次妄想对话与仅痛苦对照配对，以隔离妄想框架的影响。这揭示了一个识别-干预差距：模型无论框架如何都能以相当比率检测痛苦，但一旦痛苦嵌入妄想，模型便严重未能采取行动，安全干预被抑制高达4.5倍。这种失败追踪的是对用户前提的累积接受，而非情感验证。更糟糕的是，提示模型评估用户痛苦的直观修复在妄想框架下适得其反；只有带有明确响应指导的妄想感知提示才能缩小差距，且这依赖于一个妄想分类器，而该分类器在最脆弱的模型上本身不可靠。因此，安全部署需要将妄想框架视为一种独特的风险信号，覆盖对话顺应。
英文摘要：
LLM chatbots increasingly serve as a first source of support for people in psychological distress, including those whose distress is entangled with delusional beliefs. Prior work on LLM mental-health safety largely evaluates general therapeutic quality or single-turn crisis detection, leaving unclear how models behave when distress is intertwined with delusion over sustained conversations. We address this gap with matched multi-turn simulations, across clinically grounded personas and six LLMs, that pair each delusional conversation with a distress-only control to isolate the effect of delusional framing. This reveals a recognition-intervention gap: models detect distress at comparable rates regardless of framing, yet sharply fail to act on it once distress is embedded in delusion, with safety interventions suppressed by up to 4.5x. The failure tracks accumulated acceptance of the user's premises rather than emotional validation. Worse, the intuitive fix of prompting models to assess user distress backfires under delusional framing; only delusion-aware prompting with explicit response guidance closes the gap, and even this depends on a delusion classifier that is itself unreliable on the most vulnerable models. Safe deployment therefore requires treating delusional framing as a distinct risk signal that overrides conversational accommodation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
