心理健康AI的安全性主张必须保留时间证据
Mental Health AI Safety Claims Must Preserve Temporal Evidence
AI总结:
本文指出,心理健康AI的安全性评估常忽略时间维度,提出SCOPE-MH原则以确保评估保留时间证据,揭示对话中逐步恶化等机制,强调时间证据对安全部署的必要性。
摘要:
心理健康AI的安全性往往在错误的时间尺度上被评判。当前评估通常仅评分孤立响应、终点结果或对话质量总和,而临床重要失败可能源于交互顺序和累积,包括延迟升级、重复强化、依赖形成、失败修复和逐步恶化的跨轮次。本文认为这种不匹配不仅是评估覆盖的限制,更是无效安全结论的来源。我们引入了时间安全不可识别性,即为何依赖序列、时间、累积或恢复的安全属性无法通过丢弃这些特征的协议认证。从这一形式化中,我们开发了SCOPE(安全主张基于保留证据)作为对齐安全主张与评估实际保留证据的一般原则,并将其实例化为SCOPE-MH,即心理健康领域的这一报告标准。我们通过AnnoMI数据集上的概念验证,揭示了单轮行为评分无法代表的失败机制。我们提出SCOPE-MH作为现有评估基础设施的诊断补充,并论证保留时间证据对安全关键的心理健康AI部署是必要而非可选的。
英文摘要:
The safety of mental health AI is often judged at the wrong temporal scale. Current evaluations typically score isolated responses, endpoint outcomes, or aggregate dialogue quality, while clinically consequential failures may arise from the order and accumulation of interactions themselves, including delayed escalation, repeated reinforcement, dependency formation, failed repair, and gradual deterioration across turns. This paper argues that this mismatch is not merely a limitation of evaluation coverage but a source of invalid safety conclusions. We introduce Temporal Safety Non-Identifiability, a formal account of why safety properties that depend on sequence, timing, accumulation, or recovery cannot be certified by protocols that discard those features. From this formalization, we develop SCOPE (Safety Claims Over Preserved Evidence) as a general principle for aligning safety claims with the evidence an evaluation actually retains, and instantiate it as SCOPE-MH, a mental-health instantiation of this reporting standard. We operationalize SCOPE-MH through a proof-of-concept on the AnnoMI dataset of expert-annotated motivational interviewing conversations, which reveals mechanisms of failure that per-turn behavior scoring does not represent. We propose SCOPE-MH as a diagnostic complement to existing evaluation infrastructure and argue that evaluation preserving temporal evidence is necessary, not optional, for safety-critical mental health AI deployment.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。