arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2403.15482cs.CLcs.HCcs.LG
收藏

利用大型语言模型生成多级反馈以赋能新手同侪咨询师

Multi-Level Feedback Generation with Large Language Models for Empowering Novice Peer Counselors

Stanford University(斯坦福大学);Georgia Institute of Technology(佐治亚理工学院);Carnegie Mellon University(卡内基梅隆大学)

Alicja Chaszczewicz, Raj Sanjay Shah, Ryan Louie, Bruce A Arnow, Robert Kraut, Diyi Yang

AI总结:

研究利用大型语言模型为新手同侪咨询师提供情境化多级反馈,通过构建多级反馈分类法和400次对话标注数据集,设计自我改进方法,经专家评估证明能有效降低有害低质反馈风险。

摘要:

真实的实践和定制化的反馈是培养同侪咨询师临床技能的关键过程。然而,现有的反馈提供机制在很大程度上依赖人工监督。同侪咨询师通常缺乏从经验丰富的导师那里获得详细反馈的机制,这使得他们难以支持大量使用同侪咨询的心理健康问题人群。我们的工作旨在利用大型语言模型提供情境化的多级反馈,以规模化地赋能同侪咨询师,尤其是新手。为此,我们与一组资深心理治疗督导共同设计了一套多级反馈分类法,随后构建了一个公开可用的数据集,其中包含400次情感支持对话的综合反馈标注。我们进一步在大型语言模型的基础上设计了一种自我改进方法,以增强反馈的自动生成。通过领域专家的定性和定量评估,我们证明了我们的方法能将产生潜在有害和低质量反馈的风险降至最低,这在此类高风险场景中是非常理想的。

英文摘要:

Realistic practice and tailored feedback are key processes for training peer counselors with clinical skills. However, existing mechanisms of providing feedback largely rely on human supervision. Peer counselors often lack mechanisms to receive detailed feedback from experienced mentors, making it difficult for them to support the large number of people with mental health issues who use peer counseling. Our work aims to leverage large language models to provide contextualized and multi-level feedback to empower peer counselors, especially novices, at scale. To achieve this, we co-design with a group of senior psychotherapy supervisors to develop a multi-level feedback taxonomy, and then construct a publicly available dataset with comprehensive feedback annotations of 400 emotional support conversations. We further design a self-improvement method on top of large language models to enhance the automatic generation of feedback. Via qualitative and quantitative evaluation with domain experts, we demonstrate that our method minimizes the risk of potentially harmful and low-quality feedback generation which is desirable in such high-stakes scenarios.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑