arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2204.06337cs.CL
收藏

一种训练不同类型语言模型的推文健康提及分类新方法

A Novel Approach to Train Diverse Types of Language Models for Health Mention Classification of Tweets

German Research Center for Artificial Intelligence (DFKI)(德国人工智能研究中心);TU Kaiserslautern(凯泽斯劳滕工业大学);UNSW(新南威尔士大学)

Pervaiz Iqbal Khan, Imran Razzak, Andreas Dengel, Sheraz Ahmed

摘要:

健康提及分类是指对含有疾病词的文本进行疾病检测。然而,非健康和比喻性使用疾病词汇增加了任务的挑战。最近,对抗训练作为一种正则化的手段在许多NLP任务中得到了普及。在本文中,我们提出了一种新的方法来训练语言模型的健康提到分类的推文,涉及对抗训练。我们通过使用高斯噪声在不同级别的推文示例的Transformer模型的表示中添加扰动来生成对抗性示例。此外,我们采用对比损失作为一个额外的目标函数。我们在PHM2017数据集扩展版本上评估了所提出的方法。结果表明,我们提出的方法提高了分类器的性能显着超过基线方法。此外,我们的分析表明,在较早的层添加噪声提高模型的性能,而在中间层添加噪声恶化模型的性能。最后,向最终层添加噪声比中间层添加噪声性能更好。

英文摘要:

Health mention classification deals with the disease detection in a given text containing disease words. However, non-health and figurative use of disease words adds challenges to the task. Recently, adversarial training acting as a means of regularization has gained popularity in many NLP tasks. In this paper, we propose a novel approach to train language models for health mention classification of tweets that involves adversarial training. We generate adversarial examples by adding perturbation to the representations of transformer models for tweet examples at various levels using Gaussian noise. Further, we employ contrastive loss as an additional objective function. We evaluate the proposed method on the PHM2017 dataset extended version. Results show that our proposed approach improves the performance of classifier significantly over the baseline methods. Moreover, our analysis shows that adding noise at earlier layers improves models' performance whereas adding noise at intermediate layers deteriorates models' performance. Finally, adding noise towards the final layers performs better than the middle layers noise addition.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑