噪声文本数据：基于Transformer的流行NLP模型的致命弱点
Noisy Text Data: Achilles' Heel of popular transformer based NLP models
摘要：
在过去的几年里，ML社区已经创建了许多基于Transformer架构的新NLP模型。这些模型在基准数据集上的各种NLP任务中表现出出色的性能，通常超过SOTA结果。在这种成功的鼓舞下，人们经常发现行业从业者积极尝试微调这些模型，以构建用于行业用例的NLP应用程序。然而，对于从业者用于构建工业NLP应用程序的大多数数据集，很难保证数据中存在任何噪声。虽然大多数基于Transformer的NLP模型在将学习从一个数据集转移到另一个数据集方面表现得非常好，但仍不清楚这些模型在对噪声文本进行微调时的表现。我们解决了Kumar et al.（2020）提出的开放性问题，以探索流行的基于Transformer的NLP模型对文本数据中噪声的敏感性。我们继续处理他们定义的噪音-拼写错误和错别字（这是最常见的噪音）。我们表明（通过实验结果），这些模型在最常见的NLP任务，即文本分类，文本相似性，NER，问答，文本摘要基准数据集上表现不佳。我们进一步表明，随着数据中的噪声增加，性能下降。我们的研究结果表明，人们必须在微调流行的基于Transformer的NLP模型时，改变数据集中噪声的存在。
英文摘要：
In the last few years, the ML community has created a number of new NLP models based on transformer architecture. These models have shown great performance for various NLP tasks on benchmark datasets, often surpassing SOTA results. Buoyed with this success, one often finds industry practitioners actively experimenting with fine-tuning these models to build NLP applications for industry use cases. However, for most datasets that are used by practitioners to build industrial NLP applications, it is hard to guarantee the presence of any noise in the data. While most transformer based NLP models have performed exceedingly well in transferring the learnings from one dataset to another, it remains unclear how these models perform when fine-tuned on noisy text. We address the open question by Kumar et al. (2020) to explore the sensitivity of popular transformer based NLP models to noise in the text data. We continue working with the noise as defined by them -- spelling mistakes & typos (which are the most commonly occurring noise). We show (via experimental results) that these models perform badly on most common NLP tasks namely text classification, textual similarity, NER, question answering, text summarization on benchmark datasets. We further show that as the noise in data increases, the performance degrades. Our findings suggest that one must be vary of the presence of noise in their datasets while fine-tuning popular transformer based NLP models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
