arXiv 2402.091512024-06-12cs.CLcs.LG

Chinese MentalBERT：面向中文心理健康文本分析的社交媒体领域自适应预训练

Chinese MentalBERT: Domain-Adaptive Pre-training on Social Media for Chinese Mental Health Text Analysis

School of Software Engineering, Beijing University of Technology（北京工业大学软件学院）；School of Nursing, Wuhan University（武汉大学护理学院）；Sorbonne Université（索邦大学）；CNRS（法国国家科学研究中心）；Inria（法国国家信息与自动化研究所）；Inserm（法国国家健康与医学研究院）；AP-HP（巴黎公共医疗集团）；Hôpital de la Pitié-Salpêtrière（皮蒂-萨尔皮特里医院）

Wei Zhai, Hongzhi Qi, Qing Zhao, Jianqiang Li, Ziqi Wang, Han Wang, Bing Xiang Yang, Guanghui Fu

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出Chinese MentalBERT，通过收集336万条中文社交媒体心理文本并引入心理词典掩码机制进行领域自适应预训练，在六个公开数据集上优于八个基线模型，并公开模型与代码。

摘要：

在当前环境下，心理问题普遍且广泛存在，社交媒体成为个体表达情感的重要渠道。这导致每天产生海量数据，其中负面情绪有可能引发危机情境。人们认识到需要能够进行高效分析的模型。虽然预训练语言模型已在广泛领域展现出有效性，但针对心理学等专业领域定制的预训练模型仍存在明显空白。为解决这一问题，我们从中文社交媒体平台收集了大规模数据集，并用公开可用数据集加以扩充，构建了一个包含336万条文本的综合数据库。为增强模型对心理文本分析的适用性，我们将心理词典整合到预训练掩码机制中。在现有中文语言模型的基础上，我们进行了自适应训练，开发出专门面向心理领域的模型。我们在六个公开数据集上评估了模型性能，与八个其他模型相比，该模型表现出了改进。此外，在定性比较实验中，面对被掩码的句子，我们的模型给出了与心理学相关的预测。出于对数据隐私的担忧，该数据集将不会公开。不过，我们已将预训练模型和代码向社区公开，可通过以下链接获取：https://github.com/zwzzzQAQ/Chinese-MentalBERT。

英文摘要：

In the current environment, psychological issues are prevalent and widespread, with social media serving as a key outlet for individuals to share their feelings. This results in the generation of vast quantities of data daily, where negative emotions have the potential to precipitate crisis situations. There is a recognized need for models capable of efficient analysis. While pre-trained language models have demonstrated their effectiveness broadly, there's a noticeable gap in pre-trained models tailored for specialized domains like psychology. To address this, we have collected a huge dataset from Chinese social media platforms and enriched it with publicly available datasets to create a comprehensive database encompassing 3.36 million text entries. To enhance the model's applicability to psychological text analysis, we integrated psychological lexicons into the pre-training masking mechanism. Building on an existing Chinese language model, we performed adaptive training to develop a model specialized for the psychological domain. We evaluated our model's performance across six public datasets, where it demonstrated improvements compared to eight other models. Additionally, in the qualitative comparison experiment, our model provided psychologically relevant predictions given the masked sentences. Due to concerns regarding data privacy, the dataset will not be made publicly available. However, we have made the pre-trained models and codes publicly accessible to the community via: https://github.com/zwzzzQAQ/Chinese-MentalBERT.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。