Building Trust in Mental Health Chatbots: Safety Metrics and LLM-Based Evaluation Tools
Sue & Bill Gross School of Nursing, University of California, Irvine（加州大学欧文分校苏与比尔·格罗斯护理学院）；Department of Computer Sciences, University of California, Irvine（加州大学欧文分校计算机科学系）；School of Medicine, University of California, Irvine（加州大学欧文分校医学院）；Department of Psychological Science, University of California, Irvine（加州大学欧文分校心理科学系）；Chicago School of Professional Psychology（芝加哥专业心理学院）；HealthUnity
Jung In Park, Mahyar Abbasian, Iman Azimi, Dawn T. Bounds, Angela Jun, Jaesu Han, Robert M. McCarron, Jessica Borelli, Parmida Safavi, Sanaz Mirbaha, Jia Li, Mo… 展开作者
Jung In Park, Mahyar Abbasian, Iman Azimi, Dawn T. Bounds, Angela Jun, Jaesu Han, Robert M. McCarron, Jessica Borelli, Parmida Safavi, Sanaz Mirbaha, Jia Li, Mona Mahmoudi, Carmen Wiedenhoeft, Amir M. Rahmani
Objective: This study aims to develop and validate an evaluation framework to ensure the safety and reliability of mental health chatbots, which are increasingly popular due to their accessibility, human-like interactions, and context-aware support. Materials and Methods: We created an evaluation framework with 100 benchmark questions and ideal responses, and five guideline questions for chatbot responses. This framework, validated by mental health experts, was tested on a GPT-3.5-turbo-based chatbot. Automated evaluation methods explored included large language model (LLM)-based scoring, an agentic approach using real-time data, and embedding models to compare chatbot responses against ground truth standards. Results: The results highlight the importance of guidelines and ground truth for improving LLM evaluation accuracy. The agentic method, dynamically accessing reliable information, demonstrated the best alignment with human assessments. Adherence to a standardized, expert-validated framework significantly enhanced chatbot response safety and reliability. Discussion: Our findings emphasize the need for comprehensive, expert-tailored safety evaluation metrics for mental health chatbots. While LLMs have significant potential, careful implementation is necessary to mitigate risks. The superior performance of the agentic approach underscores the importance of real-time data access in enhancing chatbot reliability. Conclusion: The study validated an evaluation framework for mental health chatbots, proving its effectiveness in improving safety and reliability. Future work should extend evaluations to accuracy, bias, empathy, and privacy to ensure holistic assessment and responsible integration into healthcare. Standardized evaluations will build trust among users and professionals, facilitating broader adoption and improved mental health support through technology.
Mapping Caregiver Needs to AI Chatbot Design: Strengths and Gaps in Mental Health Support for Alzheimer's and Dementia Caregivers
University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Indiana University Indianapolis（印第安纳大学印第安纳波利斯分校）；University of Massachusetts Amherst（马萨诸塞大学阿姆赫斯特分校）
TrustMH-Bench: A Comprehensive Benchmark for Evaluating the Trustworthiness of Large Language Models in Mental Health
Renmin University of China, China（中国人民大学）；Beijing University of Posts and Telecommunications, China（北京邮电大学）；Hefei University of Technology, China（合肥工业大学）
Wearable Meets LLM for Stress Management: A Duoethnographic Study Integrating Wearable-Triggered Stressors and LLM Chatbots for Personalized Interventions
University of Memphis（孟菲斯大学）；Virginia Tech（弗吉尼亚理工大学）
EmoAgent: Assessing and Safeguarding Human-AI Interaction for Mental Health Safety
Department of Electrical & Computer Engineering, Princeton University（普林斯顿大学电气与计算机工程系）；Department of Computer Science, Princeton University（普林斯顿大学计算机科学系）；Department of Computer Science & Engineering, University of Michigan（密歇根大学计算机科学与工程系）；Department of Data Science & Engineering, University of Michigan（密歇根大学数据科学与工程系）；Department of Philosophy, Columbia University（哥伦比亚大学哲学系）；AI Lab, Princeton University（普林斯顿大学人工智能实验室）；Chen Frontier Lab for Al and Mental Health, Tianqiao and Chrissy Chen Institute（天桥及克里斯西·陈研究所人工智能与心理健康前沿实验室）；Theta Health Inc.（Theta健康公司）
Can AI Relate: Testing Large Language Model Response for Mental Health Support
New York University（纽约大学）；Grossman School of Medicine（格罗斯曼医学院）；Massachusetts Institute of Technology（麻省理工学院）；University of California, Los Angeles（加州大学洛杉矶分校）
On the Reliability of Large Language Models to Misinformed and Demographically-Informed Prompts
Mohamed Bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）；CIMNE（国际数值方法工程中心）；University of Lleida（莱里达大学）；Dalhousie University（达尔豪斯大学）；University of Toronto（多伦多大学）；University of Ottawa（渥太华大学）
Developing and evaluating a chatbot to support maternal health care
Carnegie Mellon University（卡内基梅隆大学）；Population Council Institute（人口理事会研究所）；Sitaram Bhartia Institute of Science and Research（西塔拉姆·巴提亚科学与研究研究所）；Nivi, Inc.（Nivi公司）
Single-turn emergency psychiatric triage across 15 frontier AI chatbots
Max Planck UCL Centre for Computational Psychiatry and Ageing Research（马克斯·普朗克大学伦敦计算精神病学与衰老研究中心）；UK AI Security Institute（英国人工智能安全研究所）；Department of Experimental Psychology, University of Oxford（牛津大学实验心理学系）；Microsoft AI（微软人工智能）；Department of Psychiatry, University of Oxford（牛津大学精神病学系）
ChatCounselor: A Large Language Models for Mental Health Support
Hong Kong University（香港大学）；Hong Kong University of Science and Technology（香港科技大学）；The International Digital Economy Academy (IDEA)（国际数字经济研究院（IDEA））；Ohio State University（俄亥俄州立大学）
Structured Dialogue System for Mental Health: An LLM Chatbot Leveraging the PM+ Guidelines
Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；University of Chinese Academy of Sciences（中国科学院大学）；Institute of Software, Chinese Academy of Sciences（中国科学院软件研究所）；East China Normal University（华东师范大学）；Wuhan Research Institute of Posts and Telecommunications（武汉邮电科学研究院）
