arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2412.21022cs.LG
收藏

文本分类:神经网络、机器学习模型与预训练模型的比较

Text Classification: Neural Networks VS Machine Learning Models VS Pre-trained Models

Temple University(天普大学)

Christos Petridis

AI总结:

本文比较了文本分类中七种预训练模型、三种神经网络和三种机器学习模型的表现,并对比了 TF-IDF 与 GloVe 嵌入技术,发现 GloVe 始终优于 TF-IDF,而 BERT、DistilBERT 等预训练模型整体表现最佳。

摘要:

文本分类在当今是一项非常常见的任务,我们可以采用许多高效的方法和算法来完成它。Transformer 已经彻底改变了深度学习领域,尤其是在自然语言处理(NLP)中,并迅速扩展到其他领域,如计算机视觉、时间序列分析等。Transformer 模型最初是在机器翻译的背景下提出的,其架构依赖自注意力机制来捕捉数据序列中的复杂关系。与传统的神经网络(如循环神经网络和多层感知机)相比,它能够更有效地处理长距离依赖。在这项工作中,我们对执行文本分类的不同技术进行了比较。我们考虑了七个预训练模型、三个标准神经网络和三个机器学习模型。对于标准神经网络和机器学习模型,我们还比较了两种嵌入技术:TF-IDF 和 GloVe,后者始终优于前者。最后,我们展示了实验结果,其中 BERT 和 DistilBERT 等预训练模型始终比标准模型/算法表现更好。

英文摘要:

Text classification is a very common task nowadays and there are many efficient methods and algorithms that we can employ to accomplish it. Transformers have revolutionized the field of deep learning, particularly in Natural Language Processing (NLP) and have rapidly expanded to other domains such as computer vision, time-series analysis and more. The transformer model was firstly introduced in the context of machine translation and its architecture relies on self-attention mechanisms to capture complex relationships within data sequences. It is able to handle long-range dependencies more effectively than traditional neural networks (such as Recurrent Neural Networks and Multilayer Perceptrons). In this work, we present a comparison between different techniques to perform text classification. We take into consideration seven pre-trained models, three standard neural networks and three machine learning models. For standard neural networks and machine learning models we also compare two embedding techniques: TF-IDF and GloVe, with the latter consistently outperforming the former. Finally, we demonstrate the results from our experiments where pre-trained models such as BERT and DistilBERT always perform better than standard models/algorithms.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑