arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2202.03480cs.CLcs.LG
收藏

使用BERT模型的迁移学习进行通用垃圾邮件检测

Universal Spam Detection using Transfer Learning of BERT Model

University of Louisiana at Lafayette(路易斯安那大学拉法叶分校)

Vijay Srinivas Tida, Sonya Hsu

摘要:

深度学习Transformer模型通过基于自我注意力机制对文本数据进行训练而变得重要。这篇手稿展示了一种新的通用垃圾邮件检测模型,该模型使用预训练的Google双向编码器表示Transformers(BERT),基于四个数据集的未封装模型,通过在实时场景中有效地分类火腿或垃圾邮件。针对Enron、Spamassain、Lingspam和Spamtext消息分类数据集使用不同的方法来单独训练模型,其中在四个数据集上获得具有可接受性能的单个模型。通用垃圾邮件检测模型(USDM)使用四个数据集进行训练,并利用每个模型的超参数。组合模型分别使用来自这四个模型的相同超参数进行微调。当每个模型使用其相应的数据集时,单个模型的F1得分为0.9或以上。总体准确率达到97%,F1得分为0.96。最后,对研究结果和意义进行了讨论。

英文摘要:

Deep learning transformer models become important by training on text data based on self-attention mechanisms. This manuscript demonstrated a novel universal spam detection model using pre-trained Google's Bidirectional Encoder Representations from Transformers (BERT) base uncased models with four datasets by efficiently classifying ham or spam emails in real-time scenarios. Different methods for Enron, Spamassain, Lingspam, and Spamtext message classification datasets, were used to train models individually in which a single model was obtained with acceptable performance on four datasets. The Universal Spam Detection Model (USDM) was trained with four datasets and leveraged hyperparameters from each model. The combined model was finetuned with the same hyperparameters from these four models separately. When each model using its corresponding dataset, an F1-score is at and above 0.9 in individual models. An overall accuracy reached 97%, with an F1 score of 0.96. Research results and implications were discussed.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑