arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2408.08261cs.CL
收藏

mhGPT:一种用于心理健康文本分析的轻量级生成式预训练Transformer

mhGPT: A Lightweight Generative Pre-Trained Transformer for Mental Health Text Analysis

Children’s National Hospital(儿童国家医院);George Washington University(乔治·华盛顿大学)

Dae-young Kim, Rebecca Hwa, Muhammad Mahbubur Rahman

AI总结:

本研究提出轻量级生成式预训练Transformer模型mhGPT,通过整合多元心理健康数据、自定义分词器并优化小模型架构,在低算力、少数据条件下实现了优于大模型的心理健康文本分析性能,助力低资源地区的AI心理健康护理发展。

摘要:

本文介绍了mhGPT,这是一款在心理健康相关社交媒体内容与PubMed文章上训练的轻量级生成式预训练Transformer。研究针对特定心理健康任务对其进行微调,在有限硬件约束下开展评估,并与MentaLLaMA、Gemma等当前最优模型进行对比。尽管mhGPT仅含19.8亿参数,且仅使用了5%的数据集,其表现仍优于更大规模的模型,且性能与使用海量数据训练的模型相当。该研究的核心贡献包括整合多元心理健康数据、构建自定义分词器,以及针对低资源场景优化小型模型架构。这项研究有望推动AI驱动的心理健康护理发展,尤其适用于算力有限的地区。

英文摘要:

This paper introduces mhGPT, a lightweight generative pre-trained transformer trained on mental health-related social media and PubMed articles. Fine-tuned for specific mental health tasks, mhGPT was evaluated under limited hardware constraints and compared with state-of-the-art models like MentaLLaMA and Gemma. Despite having only 1.98 billion parameters and using just 5% of the dataset, mhGPT outperformed larger models and matched the performance of models trained on significantly more data. The key contributions include integrating diverse mental health data, creating a custom tokenizer, and optimizing a smaller architecture for low-resource settings. This research could advance AI-driven mental health care, especially in areas with limited computing power.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑