arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2009.07238cs.CLcs.AIcs.LG
收藏

应用现成BERT的经验教训:没有灵丹妙药

Lessons Learned from Applying off-the-shelf BERT: There is no Silver Bullet

Ben-Gurion University of the Negev(内盖夫本-古里安大学)

Victor Makarenkov, Lior Rokach

摘要:

NLP领域的挑战之一是训练大型分类模型,这是一项既困难又繁琐的任务。当GPU硬件不可用时,这就更难了。预训练和现成的词嵌入、模型和模块的可用性增加,旨在简化训练大型模型的过程并实现有竞争力的性能。我们探索了现成的BERT模型的使用,并分享了我们的实验结果,并将其结果与LSTM网络和更简单的基线进行了比较。我们表明,BERT的复杂性和计算成本是不是一个保证增强预测性能的分类任务在手。

英文摘要:

One of the challenges in the NLP field is training large classification models, a task that is both difficult and tedious. It is even harder when GPU hardware is unavailable. The increased availability of pre-trained and off-the-shelf word embeddings, models, and modules aim at easing the process of training large models and achieving a competitive performance. We explore the use of off-the-shelf BERT models and share the results of our experiments and compare their results to those of LSTM networks and more simple baselines. We show that the complexity and computational cost of BERT is not a guarantee for enhanced predictive performance in the classification tasks at hand.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑