LLM治疗师行为评估的计算框架
A Computational Framework for Behavioral Assessment of LLM Therapists
AI总结：
针对LLM治疗师缺乏系统性评估的问题，提出BOLT计算框架，通过13种心理治疗方法定量测量其对话行为并与高低质量人类治疗对比，揭示其行为特点及需改进之处。
摘要：
ChatGPT等大型语言模型（LLMs）的出现，提升了将其用作治疗师以应对心理健康挑战和普遍存在的医疗可及性不足问题的兴趣。然而，专家强调，对基于LLM的心理健康干预进行系统性评估以准确评估其能力和局限性至关重要。在此，我们提出BOLT，一个概念验证计算框架，用于系统性评估LLM治疗师的对话行为。我们通过上下文学习方法，在13种心理治疗方法中定量测量LLM行为。然后，我们将LLM的行为与高质量和低质量的人类治疗进行比较。基于动机性访谈疗法的分析显示，LLM的行为往往更类似于低质量治疗中常见的行为，而非高质量治疗，例如当客户分享情绪时提供更高程度的问题解决建议。但与低质量治疗不同，LLM对客户的需求和优势的反思显著更多。我们的研究结果提醒，LLM治疗师仍需进一步研究以实现一致的高质量护理。
英文摘要：
The emergence of large language models (LLMs) like ChatGPT has increased interest in their use as therapists to address mental health challenges and the widespread lack of access to care. However, experts have emphasized the critical need for systematic evaluation of LLM-based mental health interventions to accurately assess their capabilities and limitations. Here, we propose BOLT, a proof-of-concept computational framework to systematically assess the conversational behavior of LLM therapists. We quantitatively measure LLM behavior across 13 psychotherapeutic approaches with in-context learning methods. Then, we compare the behavior of LLMs against high- and low-quality human therapy. Our analysis based on Motivational Interviewing therapy reveals that LLMs often resemble behaviors more commonly exhibited in low-quality therapy rather than high-quality therapy, such as offering a higher degree of problem-solving advice when clients share emotions. However, unlike low-quality therapy, LLMs reflect significantly more upon clients' needs and strengths. Our findings caution that LLM therapists still require further research for consistent, high-quality care.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
