arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.21638cs.CL
收藏

多维评估与可解释反馈:基于LLM的文本心理咨询中对来访者抵触的评估

Multi-dimensional Assessment and Explainable Feedback for Counselor Responses to Client Resistance in Text-based Counseling with LLMs

Zhejiang University(浙江大学);Westlake University(西湖大学)

Anqi Li, Ruihan Wang, Zhaoming Chen, Yuqian Chen, Yu Lu, Yi Zhu, Yuan Xie, Zhenzhong Lan

AI总结:

本文提出多维评估与可解释反馈框架,用于评估LLM在文本心理咨询中对来访者抵触的响应质量,实验表明其在区分沟通机制质量上优于现有模型。

摘要:

有效应对来访者的抵触是心理咨询服务中的复杂临床技能,但从业者往往缺乏及时且可扩展的监督反馈来改进其方法。尽管当前NLP研究已考察了整体咨询质量和一般治疗技能,但未能提供对高风险时刻中来访者表现出抵触的细致评估。在本工作中,我们提出了一种全面的流程,用于多维度评估人类咨询师的干预,特别是针对文本疗法中来访者的抵触。我们引入了一个理论驱动的框架,将咨询师的回应分解为四种不同的沟通机制。利用这一框架,我们整理并分享了一个由专家标注的现实世界咨询片段数据集,将咨询师-来访者互动与专业评分和解释性理由配对。使用这些数据,我们对Llama-3.1-8B-Instruct模型进行了全参数指令微调,以建模对响应质量的细致评估判断并生成解释。实验结果表明,我们的方法能够有效区分不同沟通机制的质量(77-81% F1),明显优于GPT-4o和Claude-3.5-Sonnet(45-59% F1)。此外,模型生成的高质量解释与专家参考高度一致,并获得了人类专家几乎满分的评价(2.8-2.9/3.0)。与43名咨询师的受控实验进一步证实,接收这些AI生成的反馈显著提高了咨询师应对来访者抵触的有效能力。

英文摘要:

Effectively addressing client resistance is a sophisticated clinical skill in psychological counseling, yet practitioners often lack timely and scalable supervisory feedback to refine their approaches. Although current NLP research has examined overall counseling quality and general therapeutic skills, it fails to provide granular evaluations of high-stakes moments where clients exhibit resistance. In this work, we present a comprehensive pipeline for the multi-dimensional evaluation of human counselors' interventions specifically targeting client resistance in text-based therapy. We introduce a theory-driven framework that decomposes counselor responses into four distinct communication mechanisms. Leveraging this framework, we curate and share an expert-annotated dataset of real-world counseling excerpts, pairing counselor-client interactions with professional ratings and explanatory rationales. Using this data, we perform full-parameter instruction tuning on a Llama-3.1-8B-Instruct backbone to model fine-grained evaluative judgments of response quality and generate explanations underlying. Experimental results show that our approach can effectively distinguish the quality of different communication mechanisms (77-81% F1), substantially outperforming GPT-4o and Claude-3.5-Sonnet (45-59% F1). Moreover, the model produces high-quality explanations that closely align with expert references and receive near-ceiling ratings from human experts (2.8-2.9/3.0). A controlled experiment with 43 counselors further confirms that receiving these AI-generated feedback significantly improves counselors' ability to respond effectively to client resistance.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑