共情对话回复的多维评估
Multi-dimensional Evaluation of Empathetic Dialog Responses
AI总结：
提出多维共情评估框架，结合说话者表达意图与倾听者感知共情，并基于Flan-T5指令微调分类器实现低成本自动评估，优于GPT-4等提示方法。
摘要：
共情对于有效且令人满意的对话沟通至关重要。先前测量对话共情的研究主要集中在表达的沟通意图上，即共情的表达方式。然而，这些工作忽略了对话也是一种涉及说话者和倾听者的协作过程。相比之下，我们提出了一个多维共情评估框架，用于测量从说话者视角出发的表达意图和从倾听者视角出发的感知共情。我们应用该分析框架来检查内部客服对话。我们发现这两个维度（表达的意图类型和感知的共情）相互关联，且感知的共情与对话满意度水平高度相关。为了降低标注成本，我们探索了自动测量对话共情的不同方案：提示LLM和训练基于语言模型的分类器。实验表明，使用GPT-4和Flan系列等流行模型进行提示的方法在公开和内部数据集上表现相对较差。相比之下，基于Flan-T5系列模型的指令微调分类器优于先前的工作和有竞争力的基线。我们进行了详细的消融研究，以深入了解指令微调方法的出色表现。
英文摘要：
Empathy is critical for effective and satisfactory conversational communication. Prior efforts to measure conversational empathy mostly focus on expressed communicative intents -- that is, the way empathy is expressed. Yet, these works ignore the fact that conversation is also a collaboration involving both speakers and listeners. In contrast, we propose a multi-dimensional empathy evaluation framework to measure both \emph{expressed intents from the speaker's perspective} and \emph{perceived empathy from the listener's perspective}. We apply our analytical framework to examine internal customer-service dialogues. We find the two dimensions (expressed intent types and perceived empathy) are inter-connected, while perceived empathy has high correlations with dialogue satisfaction levels. To reduce the annotation cost, we explore different options to automatically measure conversational empathy: prompting LLMs and training language model-based classifiers. Our experiments show that prompting methods with even popular models like GPT-4 and Flan family models perform relatively poorly on both public and our internal datasets. In contrast, instruction-finetuned classifiers based on Flan-T5 family models outperform prior works and competitive baselines. We conduct a detailed ablation study to give more insights into instruction finetuning method's strong performance.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
