TrustMH-Bench: 一个用于评估大语言模型在心理健康领域可信度的综合基准
TrustMH-Bench: A Comprehensive Benchmark for Evaluating the Trustworthiness of Large Language Models in Mental Health
AI总结：
TrustMH-Bench提出了一种综合基准，用于评估大语言模型在心理健康领域的可信度，揭示了现有模型在关键维度上的不足。
摘要：
尽管大型语言模型（LLMs）在提供可及性心理健康支持方面展现出显著潜力，但其在实际部署中由于领域高风险和安全敏感性质，引发了关键的可信度问题。现有针对通用目的LLMs的评估范式未能捕捉到心理健康特定要求，突显了迫切需要优先考虑和增强其可信度的必要性。为此，我们提出了TrustMH-Bench，一个系统框架，旨在系统性地量化心理健康LLMs的可信度。通过建立领域特定规范到定量评估指标的深度映射，TrustMH-Bench在可靠性、危机识别与升级、安全性、公平性、隐私、鲁棒性、反阿谀和伦理等八个核心支柱上评估模型。我们对六种通用目的LLMs和六种专门的心理健康模型进行了广泛实验。实验结果表明，评估的模型在各种心理健康场景的可信度维度上表现不佳，揭示了显著的缺陷。值得注意的是，即使是一般强大的模型（例如GPT-5.1）也无法在所有维度上保持一致的高性能。因此，系统性地提高LLMs的可信度已成为一项关键任务。我们的数据和代码已发布。
英文摘要：
While Large Language Models (LLMs) demonstrate significant potential in providing accessible mental health support, their practical deployment raises critical trustworthiness concerns due to the domains high-stakes and safety-sensitive nature. Existing evaluation paradigms for general-purpose LLMs fail to capture mental health-specific requirements, highlighting an urgent need to prioritize and enhance their trustworthiness. To address this, we propose TrustMH-Bench, a holistic framework designed to systematically quantify the trustworthiness of mental health LLMs. By establishing a deep mapping from domain-specific norms to quantitative evaluation metrics, TrustMH-Bench evaluates models across eight core pillars: Reliability, Crisis Identification and Escalation, Safety, Fairness, Privacy, Robustness, Anti-sycophancy, and Ethics. We conduct extensive experiments across six general-purpose LLMs and six specialized mental health models. Experimental results indicate that the evaluated models underperform across various trustworthiness dimensions in mental health scenarios, revealing significant deficiencies. Notably, even generally powerful models (e.g., GPT-5.1) fail to maintain consistently high performance across all dimensions. Consequently, systematically improving the trustworthiness of LLMs has become a critical task. Our data and code are released.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
