Large language models (LLMs) are increasingly applied in mental health support systems, where reliable recognition of high-risk states such as suicidal ideation and self-harm is safety-critical. However, existing evaluations primarily rely on aggregate performance metrics, which often obscure risk-specific failure modes and provide limited insight into model behavior in realistic, multi-turn interactions. We present MHDash, an open-source platform designed to support the development, evaluation, and auditing of AI systems for mental health applications. MHDash integrates data collection, structured annotation, multi-turn dialogue generation, and baseline evaluation into a unified pipeline. The platform supports annotations across multiple dimensions, including Concern Type, Risk Level, and Dialogue Intent, enabling fine-grained and risk-aware analysis. Our results reveal several key findings: (i) simple baselines and advanced LLM APIs exhibit comparable overall accuracy yet diverge significantly on high-risk cases; (ii) some LLMs maintain consistent ordinal severity ranking while failing absolute risk classification, whereas others achieve reasonable aggregate scores but suffer from high false negative rates on severe categories; and (iii) performance gaps are amplified in multi-turn dialogues, where risk signals emerge gradually. These observations demonstrate that conventional benchmarks are insufficient for safety-critical mental health settings. By releasing MHDash as an open platform, we aim to promote reproducible research, transparent evaluation, and safety-aligned development of AI systems for mental health support.
How effective are VLMs in assisting humans in inferring the quality of mental models from Multimodal short answers?
Department of Computer Science and Engineering, IIT Bombay（印度理工学院班加罗尔分校计算机科学与工程系）；Center for Educational Technology, IIT Bombay（印度理工学院班加罗尔分校教育技术中心）；School of Management, Mahindra University（马恒达大学管理学院）
Large Language Models Are Effective Human Annotation Assistants, But Not Good Independent Annotators
Department of Computer Science, University of Maryland（大学计算机科学系）；National Consortium for the Study of Terrorism and Responses to Terrorism（反恐与反恐响应国家研究中心）
A Comprehensive Review of Datasets for Clinical Mental Health AI Systems
Ubiquitous Knowledge Processing Lab (UKP Lab) Department of Computer Science and Hessian Center for AI (hessian.AI) Technische Universität Darmstadt（通用知识处理实验室（UKP Lab）计算机科学系和海斯赛中心（hessian.AI）德累斯顿技术大学）；National Research Center for Applied Cybersecurity ATHENE, Germany（应用网络安全国家研究中心ATHENE德国）；Department of Electrical Engineering, Indian Institute of Technology Delhi, India（电子工程系，印度德里印度理工学院）；Yardi School of Artificial Intelligence, Indian Institute of Technology Delhi, India（Yardi人工智能学院，印度德里印度理工学院）
