缓解语义漂移：通过MI对话摘要评估LLMs在心理治疗中的有效性
Mitigating Semantic Drift: Evaluating LLMs' Efficacy in Psychotherapy through MI Dialogue Summarization
AI总结：
本文通过MI对话摘要评估LLMs在心理治疗中的有效性，揭示其在复杂心理构念理解中的能力及缓解语义漂移的最佳实践。
摘要：
近年来，大型语言模型（LLMs）在通用和领域特定任务中的潜力得到了展示。然而，其缺乏敏感性、响应中的事实错误、同理心表达不一致、偏见、幻觉以及整体无法捕捉人类理解的深度和复杂性，尤其是在资源匮乏和敏感领域如心理学中，成为一个日益增长的担忧。为了解决这些挑战，我们的研究采用混合方法的方法来评估LLMs在心理治疗中的有效性。我们利用LLMs生成动机性访谈（MI）对话的精确摘要，并基于动机性访谈治疗完整性（MITI）框架的关键组成部分，即激发、协作、自主、方向、同理心和非评判态度，设计了一个两阶段的注释方案。使用专家注释的MI对话作为真实值，我们制定多类分类任务，以评估模型在渐进提示技术下的表现，包括单次提示和少量提示。我们的结果提供了对LLMs理解复杂心理构念能力的见解，并突显了在治疗环境中缓解
英文摘要：
Recent advancements in large language models (LLMs) have shown their potential across both general and domain-specific tasks. However, there is a growing concern regarding their lack of sensitivity, factual incorrectness in responses, inconsistent expressions of empathy, bias, hallucinations, and overall inability to capture the depth and complexity of human understanding, especially in low-resource and sensitive domains such as psychology. To address these challenges, our study employs a mixed-methods approach to evaluate the efficacy of LLMs in psychotherapy. We use LLMs to generate precise summaries of motivational interviewing (MI) dialogues and design a two-stage annotation scheme based on key components of the Motivational Interviewing Treatment Integrity (MITI) framework, namely evocation, collaboration, autonomy, direction, empathy, and a non-judgmental attitude. Using expert-annotated MI dialogues as ground truth, we formulate multi-class classification tasks to assess model performance under progressive prompting techniques, incorporating one-shot and few-shot prompting. Our results offer insights into LLMs' capacity for understanding complex psychological constructs and highlight best practices to mitigate ``semantic drift" in therapeutic settings. Our work contributes not only to the MI community by providing a high-quality annotated dataset to address data scarcity in low-resource domains but also critical insights for using LLMs for precise contextual interpretation in complex behavioral therapy.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
