数据损坏如何影响自然语言理解模型？GLUE数据集研究
How Does Data Corruption Affect Natural Language Understanding Models? A Study on GLUE datasets
摘要：
自然语言理解（NLU）研究的一个中心问题是，高性能是否表明模型具有强大的推理能力。我们提出了一系列广泛的控制实验，其中预先训练的语言模型暴露于已经经历了特定的腐败转换的数据。这些涉及删除特定词类的实例，并且经常导致无意义的句子。我们的结果表明，当模型在损坏的数据上进行微调或测试时，大多数GLUE任务的性能仍然很高，这表明即使在无意义的背景下，它们也会利用其他线索进行预测。我们提出的数据转换可用于评估特定数据集构成评估模型语言理解能力的适当测试平台的程度。
英文摘要：
A central question in natural language understanding (NLU) research is whether high performance demonstrates the models' strong reasoning capabilities. We present an extensive series of controlled experiments where pre-trained language models are exposed to data that have undergone specific corruption transformations. These involve removing instances of specific word classes and often lead to non-sensical sentences. Our results show that performance remains high on most GLUE tasks when the models are fine-tuned or tested on corrupted data, suggesting that they leverage other cues for prediction even in non-sensical contexts. Our proposed data transformations can be used to assess the extent to which a specific dataset constitutes a proper testbed for evaluating models' language understanding capabilities.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
