Background: Empathy is widely recognized for improving patient outcomes, including reduced pain and anxiety and improved satisfaction, and its absence can cause harm. Meanwhile, use of artificial intelligence (AI)-based chatbots in healthcare is rapidly expanding, with one in five general practitioners using generative AI to assist with tasks such as writing letters. Some studies suggest AI chatbots can outperform human healthcare professionals (HCPs) in empathy, though findings are mixed and lack synthesis.
Sources of data: We searched multiple databases for studies comparing AI chatbots using large language models with human HCPs on empathy measures. We assessed risk of bias with ROBINS-I and synthesized findings using random-effects meta-analysis where feasible, whilst avoiding double counting.
Areas of agreement: We identified 15 studies (2023-2024). Thirteen studies reported statistically significantly higher empathy ratings for AI, with only two studies situated in dermatology favouring human responses. Of the 15 studies, 13 provided extractable data and were suitable for pooling. Meta-analysis of those 13 studies, all utilising ChatGPT-3.5/4, showed a standardized mean difference of 0.87 (95% CI, 0.54-1.20) favouring AI (P < .00001), roughly equivalent to a two-point increase on a 10-point scale.
Areas of controversy: Studies relied on text-based assessments that overlook non-verbal cues and evaluated empathy through proxy raters.
Growing points: Our findings indicate that, in text-only scenarios, AI chatbots are frequently perceived as more empathic than human HCPs.
Areas timely for developing research: Future research should validate these findings with direct patient evaluations and assess whether emerging voice-enabled AI systems can deliver similar empathic advantages.
Exploring the Effects of Chatbot Anthropomorphism and Human Empathy on Human Prosocial Behavior Toward Chatbots
Computer Science, National University of Singapore（计算机科学，新加坡国立大学）；Department of Communications and New Media, National University of Singapore（通讯与新媒体系，新加坡国立大学）；New Media, National University of Singapore（新媒体，新加坡国立大学）
Building Trust in Mental Health Chatbots: Safety Metrics and LLM-Based Evaluation Tools
Sue & Bill Gross School of Nursing, University of California, Irvine（加州大学欧文分校苏与比尔·格罗斯护理学院）；Department of Computer Sciences, University of California, Irvine（加州大学欧文分校计算机科学系）；School of Medicine, University of California, Irvine（加州大学欧文分校医学院）；Department of Psychological Science, University of California, Irvine（加州大学欧文分校心理科学系）；Chicago School of Professional Psychology（芝加哥专业心理学院）；HealthUnity
Do You Understand How I Feel?: Towards Verified Empathy in Therapy Chatbots
Université Paris-Saclay, Centre National de la Recherche Scientifique（巴黎萨克雷大学，法国国家科学研究中心）；TU Wien（维也纳技术大学）；Politecnico di Milano（米兰理工学院）；Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院）
Anthropomorphism and Trust in Human-Large Language Model interactions
Brain and Creativity Institute, Dornsife College of Letters, Arts and Sciences, University of Southern California（脑与创造力研究所，文理学院，南加州大学）；USC Mrs. T.H. Chan Division of Occupational Science and Occupational Therapy, University of Southern California（USC 玛丽·T.H.陈职业科学与职业治疗 division，南加州大学）；Psychiatry and Biobehavioral Sciences, David Geffen School of Medicine, University of California, Los Angeles（精神病学与生物行为科学，大卫·格芬医学院，加州大学洛杉矶分校）；Google DeepMind, Zurich（谷歌深Mind，苏黎世）；Google Research（谷歌研究）；Brain and Creativity Institute（脑与创造力研究所）
Empathy by Design: Aligning Large Language Models for Healthcare Dialogue
Department of Public Health Sciences, The University of Texas at El Paso, USA（公共卫生科学系，德克萨斯理工大学埃尔帕索分校）；Department of Nursing, The University of Texas at El Paso, USA（护理系，德克萨斯理工大学埃尔帕索分校）；Department of Rehabilitation Sciences, The University of Texas at El Paso, USA（康复科学系，德克萨斯理工大学埃尔帕索分校）
FreezeEmpath: Efficient Training for Empathetic Spoken Chatbots with Frozen LLMs
Key Laboratory of Intelligent Information Processing, Institute of Computing Technology, Chinese Academy of Sciences（智能信息处理重点实验室，计算技术研究所，中国科学院）；State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences（人工智能安全国家重点实验室，计算技术研究所，中国科学院）；University of Chinese Academy of Sciences, Beijing, China（中国科学院大学，北京，中国）
Generative AI in Heritage Practice: Improving the Accessibility of Heritage Guidance
Edinburgh Futures Institute（爱丁堡未来研究所）；University of Edinburgh（爱丁堡大学）；Historic England（英国历史遗产委员会）；School of History, Classics & Archaeology University of Edinburgh（历史、古典学与考古学学院爱丁堡大学）
Domain-Specific Improvement on Psychotherapy Chatbot Using Assistant
Czech Technical University in Prague（布拉格捷克技术大学）；National Institute of Mental Health in Prague（布拉格国家精神卫生研究所）；The University of Hong Kong（香港大学）；Shenzhen Mental Health Centre（深圳市精神卫生中心）
On the Reliability of Large Language Models to Misinformed and Demographically-Informed Prompts
Mohamed Bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）；CIMNE（国际数值方法工程中心）；University of Lleida（莱里达大学）；Dalhousie University（达尔豪斯大学）；University of Toronto（多伦多大学）；University of Ottawa（渥太华大学）
Performance of a large language model-Artificial Intelligence based chatbot for counseling patients with sexually transmitted infections and genital diseases
University College of Medical Sciences and Guru Teg Bahadur Hospital（大学医学院与古鲁·特格·巴哈杜尔医院）；HeHealth Inc.（HeHealth公司）
Can AI Relate: Testing Large Language Model Response for Mental Health Support
New York University（纽约大学）；Grossman School of Medicine（格罗斯曼医学院）；Massachusetts Institute of Technology（麻省理工学院）；University of California, Los Angeles（加州大学洛杉矶分校）
Script-Strategy Aligned Generation: Aligning LLMs with Expert-Crafted Dialogue Scripts and Therapeutic Strategies for Psychotherapy
University of Amsterdam（阿姆斯特丹大学）；National Institute of Informatics (NII)（日本信息处理研究所）；Tilburg University（蒂尔堡大学）；Southeast University（东南大学）；Vrije Universiteit Amsterdam（阿姆斯特丹自由大学）；Utrecht University（乌得勒支大学）
Clean & Clear: Feasibility of Safe LLM Clinical Guidance
University College London（伦敦大学学院）；Wolfson Institute of Biomedical Research（生物医学研究沃尔夫森研究所）；King’s College Hospital（国王学院医院）；National Hospital for Neurology and Neurosurgery（神经病学与神经外科国家医院）；King’s College London（伦敦国王学院）
Towards culturally-appropriate conversational AI for health in the majority world: An exploratory study with citizens and professionals in Latin America
Imperial College London（帝国理工学院）；University of Cambridge（剑桥大学）；La Victoria Lab（拉维多利亚实验室）；Universidad Nacional de Córdoba（科尔多瓦国立大学）
ChatThero: An LLM-Supported Chatbot for Behavior Change and Therapeutic Support in Addiction Recovery
Center for Healthcare Organization and Implementation Research, VA Bedford Health Care（VA Bedford Health Care 医疗组织与实施研究中心）；Manning College of Information and Computer Sciences, University of Massachusetts Amherst（马萨诸塞大学阿默斯特分校信息与计算机科学学院）；Miner School of Computer and Information Sciences, University of Massachusetts Lowell（马萨诸塞大学洛厄尔分校计算机与信息科学学院）
DelusionEval: Measuring Delusion-Linked Behaviors in AI Chatbots
Stanford University（斯坦福大学）；University of Chicago（芝加哥大学）；Carnegie Mellon University（卡内基梅隆大学）；Harvard University（哈佛大学）；The University of Texas at Austin（德克萨斯大学奥斯汀分校）
Foundation Metrics for Evaluating Effectiveness of Healthcare Conversations Powered by Generative AI
University of California, Irvine（加州大学欧文分校）；Stanford University（斯坦福大学）；University of Pittsburgh（匹兹堡大学）；University of Toronto（多伦多大学）；HealthUnity；National Institute of Standards and Technology (NIST)（美国国家标准与技术研究院）
MoPHES:Leveraging on-device LLMs as Agent for Mobile Psychological Health Evaluation and Support
School of Software Engineering, Jiangxi University of Science and Technology（江西科技理工大学软件工程学院）；School of Computer Science, Shenzhen University（深圳大学计算机科学学院）
