Effectively addressing client resistance is a sophisticated clinical skill in psychological counseling, yet practitioners often lack timely and scalable supervisory feedback to refine their approaches. Although current NLP research has examined overall counseling quality and general therapeutic skills, it fails to provide granular evaluations of high-stakes moments where clients exhibit resistance. In this work, we present a comprehensive pipeline for the multi-dimensional evaluation of human counselors' interventions specifically targeting client resistance in text-based therapy. We introduce a theory-driven framework that decomposes counselor responses into four distinct communication mechanisms. Leveraging this framework, we curate and share an expert-annotated dataset of real-world counseling excerpts, pairing counselor-client interactions with professional ratings and explanatory rationales. Using this data, we perform full-parameter instruction tuning on a Llama-3.1-8B-Instruct backbone to model fine-grained evaluative judgments of response quality and generate explanations underlying. Experimental results show that our approach can effectively distinguish the quality of different communication mechanisms (77-81% F1), substantially outperforming GPT-4o and Claude-3.5-Sonnet (45-59% F1). Moreover, the model produces high-quality explanations that closely align with expert references and receive near-ceiling ratings from human experts (2.8-2.9/3.0). A controlled experiment with 43 counselors further confirms that receiving these AI-generated feedback significantly improves counselors' ability to respond effectively to client resistance.
PsyDT: Using LLMs to Construct the Digital Twin of Psychological Counselor with Personalized Counseling Style for Psychological Counseling
School of Future Technology, South China University of Technology(华南理工大学未来技术学院);School of Electronic and Information Engineering, South China University of Technology(华南理工大学电子与信息工程学院);Pazhou Lab(琶洲实验室)
Understanding the Therapeutic Relationship between Counselors and Clients in Online Text-based Counseling using LLMs
Zhejiang University(浙江大学);School of Engineering, Westlake University(西湖大学工程学院);Westlake University(西湖大学);Westlake University Research Center for Industries of the Future(西湖大学未来产业研究中心);Hangzhou Normal University(杭州师范大学)
KokoroChat: A Japanese Psychological Counseling Dialogue Dataset Collected via Role-Playing by Trained Counselors
The University of Electro-Communications(电通通讯大学);Rapport Technologies, Inc.(Rapport技术公司);iDEAR Human Support Service(iDEAR人类支持服务);Japanese Organization of Mental Health and Educational Agencies(日本心理健康与教育机构联合会)
CAMI: A Counselor Agent Supporting Motivational Interviewing through State Inference and Topic Exploration
Beijing Institute of Technology(北京理工大学);Singapore Management University(新加坡管理大学);Australian National University(澳大利亚国立大学);National Institute of Education(教育学院);Singapore University of Social Sciences(新加坡社科大学);National University of Singapore(新加坡国立大学)
Ψ-Arena: Interactive Assessment and Optimization of LLM-based Psychological Counselors with Tripartite Feedback
Central South University(中南大学);CoAI Group(CoAI小组);DCST(国防科学技术大学);IAI(人工智能院);BNRIST(北京航空航天大学);Tsinghua University(清华大学);Lingxin AI(灵犀AI);Fuxi AI Lab(伏羲AI实验室);NetEase Inc.(网易公司)
Can Large Language Models be Used to Provide Psychological Counselling? An Analysis of GPT-4-Generated Responses Using Role-play Dialogues
The University of Electro-Communications(电气通信大学);iDEAR Human Support Service(iDEAR人文支持服务);Japanese Organization of Mental Health and Educational Agencies(日本心理健康与教育机构组织)
UKP Lab, Department of Computer Science and Hessian Center for AI (hessian.AI), Technische Universität Darmstadt(德国达姆施塔特技术大学UKP实验室、计算机科学系和海森人工智能中心(hessian.AI));Zuse School ELIZA(Zuse学院ELIZA);National Research Center for Applied Cybersecurity ATHENE(应用网络安全国家研究中心ATHENE);Indian Institute of Technology Delhi(印度德里理工学院);Yardi School of Artificial Intelligence(Yardi人工智能学院);University of Louisville(路易斯维尔大学);University of Toledo(托莱多大学)
Enhancing Psychological Counseling with Large Language Model: A Multifaceted Decision-Support System for Non-Professionals
Sorbonne Université(索邦大学);Institut du Cerveau - Paris Brain Institute - ICM(巴黎脑科学院);CNRS(法国国家科学研究中心);Inria(法国国家信息与自动化研究所);Inserm(法国国家健康与医学研究院);AP-HP(巴黎公立医院集团);Hôpital de la Pitié Salpêtrière(萨尔佩特里埃医院);Beijing University of Technology(北京工业大学);Wuhan University(武汉大学);Wuhan Mental Health Center(武汉市精神卫生中心);Wuhan Hospital for Psychotherapy(武汉市心理医院)
CARE: An Explainable Computational Framework for Assessing Client-Perceived Therapeutic Alliance Using Large Language Models
Zhejiang University(浙江大学);Westlake University(西湖大学);Department of Psychology, Jing Hengyi School of Education, Hangzhou Normal University(心理系,金恒毅教育学院,杭州师范大学)
Distinct Profiles of Run-to-Run Score Reliability and Expert-Panel Alignment Across Four LLM Evaluators of Simulated Japanese-Language AI-to-AI Counseling
Japan National Institute of Occupational Safety and Health(日本国立职业安全卫生研究所);Kaze To Taiyo(凯泽・太阳);Saga Occupational Health Association(Saga职业健康协会);Department of Pharmacy, Zikei Hospital/Zikei Institute of Psychiatry(药剂科,Zikei医院/Zikei精神医学研究所);Department of Medical Welfare, Suzuka University of Medical Science(医疗福祉科, Suzuka医科大学);Graduate School of Human Sciences, Ritsumeikan University(人类科学研究生院,立命馆大学);Faculty of Nursing, National Defense Medical College(护理学部,国家防卫医疗大学);Support Center for Students with Disabilities, Aoyama Gakuin University(残疾学生支持中心,上智大学)
MAGneT: Coordinated Multi-Agent Generation of Synthetic Multi-Turn Mental Health Counseling Sessions
Ubiquitous Knowledge Processing Lab (UKP Lab), Department of Computer Science and Hessian Center for AI (hessian.AI), Technische Universität Darmstadt(德累斯顿技术大学计算机科学系、普遍知识处理实验室(UKP Lab)、黑森人工智能中心(hessian.AI));National Research Center for Applied Cybersecurity ATHENE, Germany(应用网络安全国家研究中心ATHENE,德国);Department of Electrical Engineering, Indian Institute of Technology Delhi, India(印度德里印度理工学院电气工程系);Yardi School of Artificial Intelligence, Indian Institute of Technology Delhi, India(印度德里印度理工学院人工智能学院)
Consistent Client Simulation for Motivational Interviewing-based Counseling
Beijing Institute of Technology(北京理工大学);Singapore Management University(新加坡管理大学);Australian National University(澳大利亚国立大学);ThoughtFull;Singapore Institute of Technology(新加坡科技设计大学);National Institute of Education(新加坡国立教育学院);Singapore University of Social Sciences(新加坡社科大学);National University of Singapore(新加坡国立大学)
MHSafeEval: Role-Aware Interaction-Level Evaluation of Mental Health Safety in Large Language Models
Department of Artificial Intelligence, Hanyang University(翰艺大学人工智能系);School of Computing and Information Systems, Singapore Management University(新加坡国立管理学院信息系)