分离提示元素层面的风险因素以评估心理健康LLM响应中的幻觉与遗漏
Disentangling Prompt Element Level Risk Factors for Hallucinations and Omissions in Mental Health LLM Responses
AI总结：
本文提出UTCO框架，通过系统性压力测试评估心理健康LLM响应中的幻觉与遗漏，发现上下文和语气是主要风险因素，支持将遗漏作为主要安全结果进行评估。
摘要：
心理健康问题常在非临床环境中表达，包括高压力求助场景，需安全关键指导。消费者健康信息学系统日益采用大型语言模型（LLMs）进行心理健康问答，但许多评估低估了叙述性高压力查询。我们引入UTCO（用户、主题、上下文、语气）提示构建框架，将查询表示为四个可控元素以系统性压力测试。使用2,075个UTCO生成的提示，我们评估了Llama 3.3，并标注了幻觉（伪造或错误的临床内容）和遗漏（缺失必要的或安全关键指导）。幻觉出现在6.5%的响应中，遗漏出现在13.2%的响应中，遗漏集中在危机和自杀念头提示中。在回归、元素特定匹配和相似性匹配比较中，失败最一致与上下文和语气相关，而用户背景指标在平衡后无系统差异。这些发现支持将遗漏作为主要安全结果进行评估，并超越静态基准问题集。
英文摘要：
Mental health concerns are often expressed outside clinical settings, including in high-distress help seeking, where safety-critical guidance may be needed. Consumer health informatics systems increasingly incorporate large language models (LLMs) for mental health question answering, yet many evaluations underrepresent narrative, high-distress inquiries. We introduce UTCO (User, Topic, Context, Tone), a prompt construction framework that represents an inquiry as four controllable elements for systematic stress testing. Using 2,075 UTCO-generated prompts, we evaluated Llama 3.3 and annotated hallucinations (fabricated or incorrect clinical content) and omissions (missing clinically necessary or safety-critical guidance). Hallucinations occurred in 6.5% of responses and omissions in 13.2%, with omissions concentrated in crisis and suicidal ideation prompts. Across regression, element-specific matching, and similarity-matched comparisons, failures were most consistently associated with context and tone, while user-background indicators showed no systematic differences after balancing. These findings support evaluating omissions as a primary safety outcome and moving beyond static benchmark question sets.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
