Large Language Models for Mental Health: A Multilingual Evaluation
George Mason University（乔治·马歇尔大学）；Interdisciplinary School of Doctoral Studies, University of Bucharest（布加勒斯特大学跨学科博士研究学院）；PRHLT Research Center, Universitat Politècnica de València（瓦伦西亚理工大学PRHLT研究中心）；University of Minnesota（明尼苏达大学）
Large Language Models (LLMs) have remarkable capabilities across NLP tasks. However, their performance in multilingual contexts, especially within the mental health domain, has not been thoroughly explored. In this paper, we evaluate proprietary and open-source LLMs on eight mental health datasets in various languages, as well as their machine-translated (MT) counterparts. We compare LLM performance in zero-shot, few-shot, and fine-tuned settings against conventional NLP baselines that do not employ LLMs. In addition, we assess translation quality across language families and typologies to understand its influence on LLM performance. Proprietary LLMs and fine-tuned open-source LLMs achieve competitive F1 scores on several datasets, often surpassing state-of-the-art results. However, performance on MT data is generally lower, and the extent of this decline varies by language and typology. This variation highlights both the strengths of LLMs in handling mental health tasks in languages other than English and their limitations when translation quality introduces structural or lexical mismatches.
Aalto University（阿尔托大学）；The University of Manchester（曼彻斯特大学）；Mila, Québec AI Institute（米拉魁北克人工智能研究所）；Nanyang Technological University（南洋理工大学）
Mental health is a critical issue in modern society, and mental disorders could sometimes turn to suicidal ideation without adequate treatment. Early detection of mental disorders and suicidal ideation from social content provides a potential way for effective social intervention. Recent advances in pretrained contextualized language representations have promoted the development of several domain-specific pretrained models and facilitated several downstream applications. However, there are no existing pretrained language models for mental healthcare. This paper trains and release two pretraine…
Leveraging LLMs for Translating and Classifying Mental Health Data
Department of Computer Science and Engineering, University of Ioannina（计算机科学与工程系，希腊伊奥亚纳大学）；LT3, IDLab, Universiteit Gent（LT3，IDLab，比利时根特大学）；Department of Informatics, Athens University of Economics and Business（信息系，希腊经济与商业大学）；Archimedes/AthenaRC, Greece（阿基米德/阿提卡RC，希腊）
Large Language Models for Mental Health Diagnostic Assessments: Exploring The Potential of Large Language Models for Assisting with Mental Health Diagnostic Assessments -- The Depression and Anxiety Case
Indian Institute of Research and Science, Bhopal（博帕尔印度研究与科学学院）；Indian Institute of Technology, Kharagpur（克勒格布尔印度理工学院）
Mental-LLM: Leveraging Large Language Models for Mental Health Prediction via Online Text Data
Massachusetts Institute of Technology（麻省理工学院）；University of Washington（华盛顿大学）；Rensselaer Polytechnic Institute（伦斯勒理工学院）；Stanford University（斯坦福大学）；University of Massachusetts Lowell（马萨诸塞大学洛厄尔分校）；Northeastern University（东北大学）
XToM: Exploring the Multilingual Theory of Mind for Large Language Models
HKUST, Hong Kong（香港理工大学）；NVIDIA AI Technology Center (NVAITC), USA（NVIDIA AI技术中心（NVAITC））；Shanghai Jiao Tong University, China（上海交通大学）；Technische Universtität Darmstadt, Germany（德累斯顿技术大学）；LMU Munich, Germany（慕尼黑大学）；Munich Center for Machine Learning, Germany（慕尼黑机器学习中心）
ProMind-LLM: Proactive Mental Health Care via Causal Reasoning with Sensor Data
California Institute of Technology（加州理工学院）；UCLA（加州大学洛杉矶分校）；National University of Singapore（新加坡国立大学）；Hangzhou Dianzi University（杭州电子科技大学）；Fudan University（复旦大学）
Large language models (LLM) have been successful in several natural language understanding tasks and could be relevant for natural language processing (NLP)-based mental health application research. In this work, we report the performance of LLM-based ChatGPT (with gpt-3.5-turbo backend) in three text-based mental health classification tasks: stress detection (2-class classification), depression detection (2-class classification), and suicidality detection (5-class classification). We obtained annotated social media posts for the three classification tasks from public datasets. Then ChatGPT AP…
TrustMH-Bench: A Comprehensive Benchmark for Evaluating the Trustworthiness of Large Language Models in Mental Health
Renmin University of China, China（中国人民大学）；Beijing University of Posts and Telecommunications, China（北京邮电大学）；Hefei University of Technology, China（合肥工业大学）
A Survey on Multilingual Mental Disorders Detection from Social Media Data
Università della Svizzera italiana（瑞士大学）；Interdisciplinary School of Doctoral Studies, University of Bucharest（布加勒斯特大学跨学科博士研究学院）；PRHLT Research Center, Universitat Politècnica de València（瓦伦西亚理工大学PRHLT研究中心）；George Mason University（乔治·梅森大学）；Lancaster University（兰卡斯特大学）
Georgetown University（乔治城大学）；Ghent University（根特大学）；Max Planck Institute for Informatics（马克斯·普朗克信息学研究所）
Mental health is a significant and growing public health concern. As language usage can be leveraged to obtain crucial insights into mental health conditions, there is a need for large-scale, labeled, mental health-related datasets of users who have been diagnosed with one or more of such conditions. In this paper, we investigate the creation of high-precision patterns to identify self-reported diagnoses of nine different mental health conditions, and obtain high-quality labeled data without the need for manual labelling. We introduce the SMHD (Self-reported Mental Health Diagnoses) dataset an…
Google Translate Error Analysis for Mental Healthcare Information: Evaluating Accuracy, Comprehensibility, and Implications for Multilingual Healthcare Communication
University of Surrey（萨里大学）；University of Amsterdam（阿姆斯特丹大学）
Mental health remains a significant challenge of public health worldwide. With increasing popularity of online platforms, many use the platforms to share their mental health conditions, express their feelings, and seek help from the community and counselors. Some of these platforms, such as Reachout, are dedicated forums where the users register to seek help. Others such as Reddit provide subreddits where the users publicly but anonymously post their mental health distress. Although posts are of varying length, it is beneficial to provide a short, but informative summary for fast processing by…
Stony Brook University（石溪大学）；Stanford University（斯坦福大学）；University of Pennsylvania（宾夕法尼亚大学）
Compared to physical health, population mental health measurement in the U.S. is very coarse-grained. Currently, in the largest population surveys, such as those carried out by the Centers for Disease Control or Gallup, mental health is only broadly captured through "mentally unhealthy days" or "sadness", and limited to relatively infrequent state or metropolitan estimates. Through the large scale analysis of social media data, robust estimation of population mental health is feasible at much higher resolutions, up to weekly estimates for counties. In the present work, we validate a pipeline t…
Georgetown University（乔治城大学）；Ghent University（根特大学）；Max Planck Institute for Informatics（马克斯·普朗克信息学研究所）；National Institutes of Health（美国国立卫生研究院）
Self-reported diagnosis statements have been widely employed in studying language related to mental health in social media. However, existing research has largely ignored the temporality of mental health diagnoses. In this work, we introduce RSDD-Time: a new dataset of 598 manually annotated self-reported depression diagnosis posts from Reddit that include temporal information about the diagnosis. Annotations include whether a mental health condition is present and how recently the diagnosis happened. Furthermore, we include exact temporal spans that relate to the date of diagnosis. This infor…
Johns Hopkins University（约翰斯·霍普金斯大学）；Microsoft Research（微软研究院）；University of Copenhagen（哥本哈根大学）
We introduce initial groundwork for estimating suicide risk and mental health in a deep learning framework. By modeling multiple conditions, the system learns to make predictions about suicide risk and mental health at a low false positive rate. Conditions are modeled as tasks in a multi-task learning (MTL) framework, with gender prediction as an additional auxiliary task. We demonstrate the effectiveness of multi-task learning by comparison to a well-tuned single-task baseline with the same number of parameters. Our best MTL model predicts potential suicide attempt, as well as the presence of…
Cross-Lingual Mental Health Ontologies for Indian Languages: Bridging Patient Expression and Clinical Understanding through Explainable AI and Human-in-the-Loop Validation
University of Florida（佛罗里达大学）；University of Kansas（堪萨斯大学）；University of Maryland（马里兰大学）；IIT Kharagpur（印度理工学院卡拉格普尔分校）
North Africa's Missing Framework: NLP-Driven Mental Healthcare in Algeria and Implications for Low-resource Settings
University Mohamed El Bachir El Ibrahimi of Bordj Bou Arreridj（布吉布阿雷里吉·穆罕默德·巴希尔·易卜拉欣大学）；Laboratory of Informatics and its Applications of M’sila（姆西拉信息学及其应用实验室）；Laboratory of Materials and Electronic Systems（材料与电子系统实验室）
MentalQA: An Annotated Arabic Corpus for Questions and Answers of Mental Healthcare
College of Computing（计算机学院）；Umm Al-Qura University（乌姆·古拉大学）；King Khalid University（哈立德国王大学）；College of Computer and Information Sciences（计算机与信息科学学院）；King Saud University（沙特国王大学）
Data-driven methods for mental health treatment and surveillance have become a major focus in computational science research in the last decade. However, progress in the domain, in terms of both medical understanding and system performance, remains bounded by the availability of adequate data. Prior systematic reviews have not necessarily made it possible to measure the degree to which data-related challenges have affected research progress. In this paper, we offer an analysis specifically on the state of social media data that exists for conducting mental health research. We do so by introduc…
The mental disorder of online users is determined using social media posts. The major challenge in this domain is to avail the ethical clearance for using the user generated text on social media platforms. Academic re searchers identified the problem of insufficient and unlabeled data for mental health classification. To handle this issue, we have studied the effect of data augmentation techniques on domain specific user generated text for mental health classification. Among the existing well established data augmentation techniques, we have identified Easy Data Augmentation (EDA), conditional…
Thapar Institute of Engineering & Technology（塔帕尔工程技术学院）；University of Florida（佛罗里达大学）；The Chinese University of Hong Kong（香港中文大学）
With recent developments in digitization of clinical psychology, NLP research community has revolutionized the field of mental health detection on social media. Existing research in mental health analysis revolves around the cross-sectional studies to classify users' intent on social media. For in-depth analysis, we investigate existing classifiers to solve the problem of causal categorization which suggests the inefficiency of learning based methods due to limited training samples. To handle this challenge, we use transformer models and demonstrate the efficacy of a pre-trained transfer learn…
Many statistical models have high accuracy on test benchmarks, but are not explainable, struggle in low-resource scenarios, cannot be reused for multiple tasks, and cannot easily integrate domain expertise. These factors limit their use, particularly in settings such as mental health, where it is difficult to annotate datasets and model outputs have significant impact. We introduce a micromodel architecture to address these challenges. Our approach allows researchers to build interpretable representations that embed domain knowledge and provide explanations throughout the model's decision proc…
The University of Adelaide（阿德莱德大学）；University of Moratuwa（莫拉图瓦大学）；University of Colombo School of Computing（科伦坡大学计算学院）；University of Colombo（科伦坡大学）
People often utilise online media (e.g., Facebook, Reddit) as a platform to express their psychological distress and seek support. State-of-the-art NLP techniques demonstrate strong potential to automatically detect mental health issues from text. Research suggests that mental health issues are reflected in emotions (e.g., sadness) indicated in a person's choice of language. Therefore, we developed a novel emotion-annotated mental health corpus (EmoMent), consisting of 2802 Facebook posts (14845 sentences) extracted from two South Asian countries - Sri Lanka and India. Three clinical psycholog…
University of Bucharest（布加勒斯特大学）；Rochester Institute of Technology（罗切斯特理工学院）
In this paper, we analyze the interplay between the use of offensive language and mental health. We acquired publicly available datasets created for offensive language identification and depression detection and we train computational models to compare the use of offensive language in social media posts written by groups of individuals with and without self-reported depression diagnosis. We also look at samples written by groups of individuals whose posts show signs of depression according to recent related studies. Our analysis indicates that offensive language is more frequently used in the …
Pattern Recognition Center, WeChat AI, Tencent Inc（腾讯公司微信人工智能模式识别中心）；University of Chinese Academy of Sciences（中国科学院大学）
Previous researches on dialogue system assessment usually focus on the quality evaluation (e.g. fluency, relevance, etc) of responses generated by the chatbots, which are local and technical metrics. For a chatbot which responds to millions of online users including minors, we argue that it should have a healthy mental tendency in order to avoid the negative psychological impact on them. In this paper, we establish several mental health assessment dimensions for chatbots (depression, anxiety, alcohol addiction, empathy) and introduce the questionnaire-based mental health assessment methods. We…
