arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1907.02581cs.CL
收藏

社交媒体帖子风险分类的迁移学习模型评价研究

Transfer Learning for Risk Classification of Social Media Posts: Model Evaluation Study

Campbell Family Mental Health Research Institute, Centre for Addiction and Mental Health(坎贝尔家族心理健康研究所,成瘾与心理健康中心);Krembil Centre for Neuroinformatics, Centre for Addiction and Mental Health(克伦比尔神经信息学中心,成瘾与心理健康中心);University of Toronto(多伦多大学);Princess Margaret Cancer Centre, University Health Network(玛格丽特公主癌症中心,大学健康网络)

Derek Howard, Marta Maslej, Justin Lee, Jacob Ritchie, Geoffrey Woollard, Leon French

摘要:

精神疾病影响着世界人口的很大一部分。在线心理健康论坛可以为那些受影响的人提供支持性环境,并且还可以生成大量数据,这些数据可以被挖掘以使用机器学习方法来预测心理健康状态。我们对社交媒体帖子的文本特征表示的多种方法进行了基准测试,并将其下游使用与自动机器学习(AutoML)工具进行了比较,以筛选内容以引起版主的注意。我们使用了从Reachout.com论坛收集的CLPsych 2017共享任务中的1588个标记帖子(Milne等人,2019年)。帖子使用基于词典的工具表示,包括VADER,Empath,LIWC,还使用预训练的人工神经网络模型,包括DeepMoji,通用句子编码器和GPT-1。我们使用TPOT和auto-sklearn作为AutoML工具来生成分类器来分类帖子。表现最好的系统使用了来自GPT-1模型的功能,该模型在Reachout.com的15万多个未标记的帖子上进行了微调。我们的顶级系统的宏观平均F1得分为0.572,为CLPsych 2017任务提供了新的最先进的结果。这是在没有来自元数据或先前帖子的额外信息的情况下实现的。错误分析显示,这个顶级系统经常忽略绝望的表达。我们还提出了可视化,以帮助理解学习分类。我们表明,迁移学习是一种有效的策略,预测风险相对较少的标记数据。我们注意到,当大量未标记文本可用时,预训练语言模型的微调提供了进一步的收益。

英文摘要:

Mental illness affects a significant portion of the worldwide population. Online mental health forums can provide a supportive environment for those afflicted and also generate a large amount of data which can be mined to predict mental health states using machine learning methods. We benchmark multiple methods of text feature representation for social media posts and compare their downstream use with automated machine learning (AutoML) tools to triage content for moderator attention. We used 1588 labeled posts from the CLPsych 2017 shared task collected from the Reachout.com forum (Milne et al., 2019). Posts were represented using lexicon based tools including VADER, Empath, LIWC and also used pre-trained artificial neural network models including DeepMoji, Universal Sentence Encoder, and GPT-1. We used TPOT and auto-sklearn as AutoML tools to generate classifiers to triage the posts. The top-performing system used features derived from the GPT-1 model, which was finetuned on over 150,000 unlabeled posts from Reachout.com. Our top system had a macro averaged F1 score of 0.572, providing a new state-of-the-art result on the CLPsych 2017 task. This was achieved without additional information from meta-data or preceding posts. Error analyses revealed that this top system often misses expressions of hopelessness. We additionally present visualizations that aid understanding of the learned classifiers. We show that transfer learning is an effective strategy for predicting risk with relatively little labeled data. We note that finetuning of pretrained language models provides further gains when large amounts of unlabeled text is available.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑