With the emergence of large language models (LLMs), investigating if they can surpass humans in areas such as emotion recognition and empathetic responding has become a focal point of research. This paper presents a comprehensive study exploring the empathetic responding capabilities of four state-of-the-art LLMs: GPT-4, LLaMA-2-70B-Chat, Gemini-1.0-Pro, and Mixtral-8x7B-Instruct in comparison to a human baseline. We engaged 1,000 participants in a between-subjects user study, assessing the empathetic quality of responses generated by humans and the four LLMs to 2,000 emotional dialogue prompts meticulously selected to cover a broad spectrum of 32 distinct positive and negative emotions. Our findings reveal a statistically significant superiority of the empathetic responding capability of LLMs over humans. GPT-4 emerged as the most empathetic, marking approximately 31% increase in responses rated as "Good" compared to the human benchmark. It was followed by LLaMA-2, Mixtral-8x7B, and Gemini-Pro, which showed increases of approximately 24%, 21%, and 10% in "Good" ratings, respectively. We further analyzed the response ratings at a finer granularity and discovered that some LLMs are significantly better at responding to specific emotions compared to others. The suggested evaluation framework offers a scalable and adaptable approach for assessing the empathy of new LLMs, avoiding the need to replicate this study's findings in future research.
The Hong Kong University of Science and Technology（香港科技大学）；EMOS Technologies Inc.（EMOS科技公司）
在本文中，我们提出了一个端到端的移情会话代理CAiRE。我们的系统适应TransferTransfo（Wolf等人，2019）学习方法，微调具有多任务目标的大规模预训练语言模型：响应语言建模，响应预测和对话情感检测。我们在最近提出的移情对话数据集（Rashkin et al.，2019），实验结果表明，CAiRE在对话情感检测和移情反应生成方面达到了最先进的性能。
FreezeEmpath: Efficient Training for Empathetic Spoken Chatbots with Frozen LLMs
Key Laboratory of Intelligent Information Processing, Institute of Computing Technology, Chinese Academy of Sciences（智能信息处理重点实验室，计算技术研究所，中国科学院）；State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences（人工智能安全国家重点实验室，计算技术研究所，中国科学院）；University of Chinese Academy of Sciences, Beijing, China（中国科学院大学，北京，中国）
Sibyl: Empowering Empathetic Dialogue Generation in Large Language Models via Sensible and Visionary Commonsense Inference
Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；School of Cyber Security, University of Chinese Academy of Sciences（中国科学院大学网络空间安全学院）；WeChat AI, Tencent Inc（腾讯微信人工智能）；Meituan Group（美团集团）
Guiding Language Models to Be More Empathetic: Culturally Sensitive Mental Health Advice Generation Through Human-LLM Collaboration
Ahsanullah University of Science and Technology（阿山努拉科技大学）；Bangladesh University of Business and Technology（孟加拉国商业与技术大学）；The University of New South Wales（新南威尔士大学）；Jashore University of Science and Technology（贾绍尔科技大学）；American International University - Bangladesh（孟加拉国美国国际大学）
Synth-Empathy: Towards High-Quality Synthetic Empathy Data
Peking University（北京大学）；University of Chinese Academy of Sciences（中国科学院大学）；Shanghai AI Laboratory（上海人工智能实验室）；Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）
Talk With Human-like Agents: Empathetic Dialogue Through Perceptible Acoustic Reception and Reaction
School of Computer Science and Technology, University of Science and Technology of China（中国科学技术大学计算机科学与技术学院）；State Key Laboratory of Cognitive Intelligence（认知智能国家重点实验室）；School of Data Science, University of Science and Technology of China（中国科学技术大学数据科学学院）；Tencent Youtu Lab（腾讯优图实验室）
EmplifAI: a Fine-grained Dataset for Japanese Empathetic Medical Dialogues in 28 Emotion Labels
Kyoto Institute of Technology, Japan（京都技术大学）；National Institute of Information and Communications Technology (NICT), Japan（信息通信技术国家研究所）；Kyoto University, Japan（京都大学）；Nara Institute of Science and Technology, Japan（奈良科学技术大学）
An Iterative Associative Memory Model for Empathetic Response Generation
College of Computer and Data Science, Fuzhou University（福州大学计算机与数据科学学院）；Leiden University（莱顿大学）；School of Computer Science and Technology, Harbin Institute of Technology, Shenzhen（哈尔滨工业大学（深圳）计算机科学与技术学院）；SmartMore（思谋科技）；College of Computer Science and Technology, Chongqing University of Technology（重庆理工大学计算机科学与工程学院）
WiseMind: a knowledge-guided multi-agent framework for accurate and empathetic psychiatric diagnosis
College of Biomedical Engineering（生物医学工程学院）；Fudan University（复旦大学）；Department of Electrical and Computer Engineering（电气与计算机工程系）；University of Alberta（阿尔伯塔大学）；University of Virginia（弗吉尼亚大学）；McIntire School of Commerce（麦金尼商学院）；School of Data Science（数据科学学院）；Department of Psychiatry（精神病学系）
ReflectDiffu:Reflect between Emotion-intent Contagion and Mimicry for Empathetic Response Generation via a RL-Diffusion Framework
University of Shanghai for Science and Technology（上海理工大学）；East China Normal University（华东师范大学）；Nanjing University of Information Science and Technology（南京信息工程大学）；Macquarie University（麦考瑞大学）
PRIDE: Privileged Information-enhanced Distillation for Empathetic Dialogue Generation
Anhui Robot Technology Standard Innovation Base, School of Computer Science and Technology, University of Science and Technology of China（中国科学技术大学计算机科学与技术学院安徽机器人技术标准创新基地）
EmoStage: A Framework for Accurate Empathetic Response Generation via Perspective-Taking and Phase Recognition
The University of Electro-Communications（电气通信大学）；iDEAR Human Support Service（iDEAR人类支持服务）；Japanese Organization of Mental Health and Educational Agencies（日本心理健康与教育机构组织）
Efficient-Empathy: Towards Efficient and Effective Selection of Empathy Data
Peking University（北京大学）；University of Chinese Academy of Sciences（中国科学院大学）；Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）
Towards Empathetic Open-domain Conversation Models: a New Benchmark and Dataset
Paul G. Allen School of Computer Science & Engineering, University of Washington（华盛顿大学保罗·G·艾伦计算机科学与工程学院）；Facebook AI Research（Facebook人工智能研究（Facebook AI Research））
