arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2604.05795cs.CL
收藏

衡量重要性!!评估心理健康对话中的治疗原则

Measuring What Matters!! Assessing Therapeutic Principles in Mental-Health Conversation

IIIT Delhi(印度德里理工学院);MBZUAI(马尔科姆·比尔人工智能研究所);Wysa

Abdullah Mazhar, Het Riteshkumar Shah, Aseem Srivastava, Smriti Joshi, Md Shad Akhtar

AI总结:

本文提出CARE框架,通过多阶段方法评估AI生成的治疗回应是否符合临床标准,实验显示其在F-1分数上优于基线模型,展示了结构化推理和上下文建模的重要性。

摘要:

随着大语言模型在心理健康应用中的普及,需要原则性的评估框架来评估其与心理治疗最佳实践的一致性。本文提出CARE框架,结合对话上下文、对比示例检索和知识蒸馏的链式推理,通过细粒度序数评分评估治疗原则。实验表明,CARE在F-1分数上优于基线模型,展示了结构化推理和上下文建模的重要性,同时验证了其在领域转移下的鲁棒性。

英文摘要:

The increasing use of large language models in mental health applications calls for principled evaluation frameworks that assess alignment with psychotherapeutic best practices beyond surface-level fluency. While recent systems exhibit conversational competence, they lack structured mechanisms to evaluate adherence to core therapeutic principles. In this paper, we study the problem of evaluating AI-generated therapist-like responses for clinically grounded appropriateness and effectiveness. We assess each therapists utterance along six therapeutic principles: non-judgmental acceptance, warmth, respect for autonomy, active listening, reflective understanding, and situational appropriateness using a fine-grained ordinal scale. We introduce FAITH-M, a benchmark annotated with expert-assigned ordinal ratings, and propose CARE, a multi-stage evaluation framework that integrates intra-dialogue context, contrastive exemplar retrieval, and knowledge-distilled chain-of-thought reasoning. Experiments show that CARE achieves an F-1 score of 63.34 versus the strong baseline Qwen3 F-1 score of 38.56 which is a 64.26 improvement, which also serves as its backbone, indicating that gains arise from structured reasoning and contextual modeling rather than backbone capacity alone. Expert assessment and external dataset evaluations further demonstrate robustness under domain shift, while highlighting challenges in modelling implicit clinical nuance. Overall, CARE provides a clinically grounded framework for evaluating therapeutic fidelity in AI mental health systems.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑