Large Language Models (LLMs) offer significant potential for delivering health information. However, their reliability in low-resource contexts remains uncertain. This study evaluates GPT-4, Gemini Pro, Llama~3, and Mistral-7B on health crisis-related enquiries concerning COVID-19, dengue, the Nipah virus, and Chikungunya in the low-resource context of Bangladesh. We constructed a question--answer dataset from authoritative sources and assessed model outputs through semantic similarity, expert-model cross-evaluation, and Natural Language Inference (NLI). Findings highlight both the strengths and limitations of LLMs in representing epidemiological history and health crisis knowledge, underscoring their promise and risks for informing policy in resource-constrained environments.
Between Help and Harm: An Evaluation of Mental Health Crisis Handling by LLMs
ELLIS Alicante(ELLIS 阿利坎特);CTU in Prague(捷克布拉格理工大学);School of Health Science, The University of Nottingham(诺丁汉大学健康科学学院);School of Psychology, Public Health and Social Care, Derby University(德比大学心理学、公共卫生与社会关怀学院);Mental Health Practitioner, Social Worker & Independent Scholar(心理健康从业者、社会工作者与独立学者);School of Computer Science & School of Medicine, The University of Nottingham(诺丁汉大学计算机科学学院与医学院)
Speech-based Psychological Crisis Assessment using LLMs
Tsinghua University(清华大学);Peking University Huilongguan Clinical Medical School(北京大学回龙guan临床医学院);WHO Collaborating Centre for Research and Training in Suicide Prevention(世界卫生组织自杀预防研究与培训协作中心)
TrustMH-Bench: A Comprehensive Benchmark for Evaluating the Trustworthiness of Large Language Models in Mental Health
Renmin University of China, China(中国人民大学);Beijing University of Posts and Telecommunications, China(北京邮电大学);Hefei University of Technology, China(合肥工业大学)
NLP for Maternal Healthcare: Perspectives and Guiding Principles in the Age of LLMs
Allen Institute for AI(艾伦人工智能研究所);Association of American Medical Colleges, Center for Health Justice(美国医学院协会健康正义中心);University of Washington(华盛顿大学);University of California, Berkeley(加州大学伯克利分校);University of California, San Francisco(加州大学旧金山分校)
Lightme: Analysing Language in Internet Support Groups for Mental Health
Commonwealth Scientific and Industrial Research Organization(澳大利亚联邦科学与工业研究组织);Australian National University(澳大利亚国立大学);Australasian Institute of Digital Health(澳大拉西亚数字健康研究所)
Toward Equitable Access: Leveraging Crowdsourced Reviews to Investigate Public Perceptions of Health Resource Accessibility
University of Pennsylvania(宾夕法尼亚大学);Renmin University of China(中国人民大学);University of Liverpool(利物浦大学);University of Michigan(密歇根大学);Northwestern University(西北大学);Massachusetts Institute of Technology(麻省理工学院);University of California, Santa Barbara(加州大学圣巴巴拉分校);Harvard Medical School(哈佛医学院);Rutgers University(新泽西州立大学罗格斯大学);University of South Florida(佛罗里达州立大学)