CBT-Bench：评估大语言模型在辅助认知行为疗法中的表现
CBT-Bench: Evaluating Large Language Models on Assisting Cognitive Behavior Therapy
AI总结：
本文提出CBT-BENCH基准，通过基础知识、认知模型理解和治疗性回应生成三个级别任务，系统评估大语言模型辅助认知行为疗法的能力，发现LLMs在知识背诵上表现良好但在复杂真实场景中不足。
摘要：
当前患者需求与可用的心理健康支持之间存在显著差距。在本文中，我们旨在全面考察使用大语言模型（LLMs）辅助专业心理治疗的潜力。为此，我们提出了一个新基准CBT-BENCH，用于系统评估认知行为疗法（CBT）辅助能力。我们在CBT-BENCH中包含三个级别的任务：I：基础CBT知识获取，任务为多项选择题；II：认知模型理解，任务包括认知扭曲分类、初级核心信念分类和细粒度核心信念分类；III：治疗性回应生成，任务为在CBT治疗对话中生成对患者言语的回应。这些任务涵盖了可通过AI辅助增强的CBT关键方面，同时勾勒出从基础知识背诵到参与真实治疗对话的能力要求层级。我们在该基准上评估了具有代表性的LLMs。实验结果表明，尽管LLMs在背诵CBT知识方面表现良好，但在需要深度分析患者认知结构并生成有效回应的复杂真实场景中表现不足，这为未来的潜在工作指明了方向。
英文摘要：
There is a significant gap between patient needs and available mental health support today. In this paper, we aim to thoroughly examine the potential of using Large Language Models (LLMs) to assist professional psychotherapy. To this end, we propose a new benchmark, CBT-BENCH, for the systematic evaluation of cognitive behavioral therapy (CBT) assistance. We include three levels of tasks in CBT-BENCH: I: Basic CBT knowledge acquisition, with the task of multiple-choice questions; II: Cognitive model understanding, with the tasks of cognitive distortion classification, primary core belief classification, and fine-grained core belief classification; III: Therapeutic response generation, with the task of generating responses to patient speech in CBT therapy sessions. These tasks encompass key aspects of CBT that could potentially be enhanced through AI assistance, while also outlining a hierarchy of capability requirements, ranging from basic knowledge recitation to engaging in real therapeutic conversations. We evaluated representative LLMs on our benchmark. Experimental results indicate that while LLMs perform well in reciting CBT knowledge, they fall short in complex real-world scenarios requiring deep analysis of patients' cognitive structures and generating effective responses, suggesting potential future work.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
